Copyright © 2000 Clark Evans, All Rights Reserved. This document may be freely copied provided it is not modified.
YAML (pronounced "yaamel") is a straight-forward machine parable data serialization format designed for human readability and interaction with scripting languages such as Perl and Python. YAML is optimized for configuration settings, log files, Internet messaging and filtering. This specification describes the serialization format, a "C" API for the parser and emitter, and several language bindings.
This specification is a working draft and reflects consensus reached by the members of the yaml-core mailing list. Any questions regarding this draft should be raised on this list. This is a draft and changes are expected, therefore implementers should closely follow this mailing list to stay up-to-date on trends and announcements.
There are many productions which may have bugs, and the MIME productions are yet to be completed, although this should not stop an implementer! Feedback is welcome.
1.1 Origin and Goals
1.2 Key Concepts
1.4 Relation to XML
2.1 Information Model
2.2.1 Character Set
2.2.3 End-of-Line Normalization
2.2.5 Escape Sequences
2.2.6 Miscellaneous Characters
2.3.2 Whitespace Folding
2.3.3 Quoted String
2.3.4 Anchor String
2.3.5 Class String
2.5.1 Simple Scalar
2.5.2 Block Scalar
2.5.4 Binary Scalar
3 Sequential Processing
3.1 Parser Interface
3.2 Emitter Interface
3.3 Interface Converter
4 Language Bindings
4.1 "C" Language
Yet Another Markup Language, abbreviated YAML, describes a class of data objects called YAML documents and partially describes the behavior of computer programs that process them.
YAML documents encode into a serialized form information having a recursive scalar, map, or list structure. YAML also includes a method to encode references and binary values. At its core, a YAML document consists of a sequence of characters, some of which are considered part of the document's content, and others that are used to indicate structure within the information stream.
A software module called a YAML parser is used to read YAML documents and provide access to their content and structure. In a similar way, a YAML emitter is used to write YAML documents, serializing their content and structure. A YAML processor is a module that provides for parser or emitter functionality or both. It is assumed that a YAML processor does its work on behalf of another module, called an application. This specification describes the interface and required behavior of an YAML processor in terms of how it must read or write YAML documents and the information it must provide or obtain from the application.
The design goals for YAML are:
YAML was designed with experience gained from the construction and deployment of Data::Denter. YAML has also enjoyed much markup language critique from SML-DEV list participants, including experience with the Minimal XML and Common XML specifications.
This specification, together with associated standards (Unicode and ISO/IEC 10646 for characters, Internet RFC 822 for mail headers and Internet RFC 2045/6 for MIME sections), provides all the information necessary to understand YAML Version 1.0 and construct computer programs to process it.
YAML uses similar type structure as Perl. In YAML, there there are three fundamental structures: scalars, maps (%) and lists (@). YAML also supports references to enable the serialization of graphs. Furthermore, each data value can be associated with a class name to allow the use of specific data types. This type structure common to many other languages and provides a solid basis for an information model. Furthermore, it enables the programmer to use their programming language's native data constructs for YAML manipulation, instead of a document object.
YAML uses similar whitespace handling as HTML. In YAML, sequences of spaces, tabs, and carriage return characters are usually folded into a single space during parse. This wonderful technique makes markup code readable by enabling indentation and word-wrapping without affecting the canonical form of the content. This folding technique can also be found in the structured headers of RFC 822.
YAML uses block scoping similar to Python. In YAML, the extent of a node is indicated by its child's nesting level, i.e., what column it is in. Block indenting provides for easy inspection of the document's structure and greatly improves readability. This block scoping is possible due to the whitespace folding technique.
YAML uses quoted strings similar to C. In YAML, scalars can be surrounded by quotes enabling escape sequences such as \n to represent a new line, \t to represent a tab, and \\ to represent the slash. Unlike C, since whitespace is folded, YAML introduces bash style "\ " to escape additional spaces that are part of the content and should not be folded. Further, the trailing \ is used as a continuation marker, allowing content to be broken into multiple lines without introducing u nwanted whitespace. Lastly, ISO 8859-1 characters can be specified using "\x3B" style escapes and UNICODE characters can be specified using a similar technique "\u003B".
The syntax of YAML is an extension of rfc822, allowing for direct usage of YAML in mail handlers. YAML also has a binary scalar style using base64 encoding.
In addition to a native in-memory load/save interface, YAML provides both a pull style input stream and a push style (SAX like) output stream interface. This enables YAML to directly support the processing of large documents, such as a transaction log, or continuous streams, such as a feed from a production machine.
Following is a simple example of an invoice represented as a YAML document. The colon is used to separate name value pairs. The percent sign following a colon indicates that the value is an mapping. The at sign indicates that the value is an ordered list.
buyer: % address : % city : Royal Oak line one : 458 Wittigen's Way line two : Suite #292 postal : 48046 state : MI family name : Dumars given name : Chris date : 12-JAN-2001 delivery: % method : UZS Express Overnight price : $45.50 comments : Mr. Dumars is frequently gone in the morning so it is best advised to try things in late afternoon. If Joe isn't around, try his house keeper, Nancy Billsmer @ (734) 338-4338. invoice : 00034843 product : @ % desc : Grade A, Leather Hide Basketball id : BL394D price : $450.00 quantity : 4 % desc : Super Hoop (tm) id : BL4438H price : $2,392.00 quantity : 1 tax : $0.00 total : $4237.50
There are many differences between YAML and the eXtensible Markup Language ("XML"). XML was designed to be backwards compatible with Standard Generalized Markup Language ("SGML") and thus had many design constraints placed on it that YAML does not share. Also XML, inheriting SGML's legacy, is designed to support structured documents, where YAML is more closely targeted at messaging with direct support for the native data structures of modern programming languages. Further, XML is a pioneer in many domains and YAML has been grown on the lessons learned by the XML community. These points aside, there are many differences.
The YAML and XML information model are starkly different. In XML, the primary construct is an attributed tree, with where each element has a ordered, named list of children and an unordered mapping of names to strings. In YAML, the primary hierarchical construct alternates between an anonymous list, named map, and scalar values. This difference is critical since YAML's model is directly supported by native data structures in most modern programming languages, where XML's model requires mapping, conventions, or an alternative programming component, a document object model.
The YAML and XML syntax vary significantly. In XML, tags are used to denote the begin and end of an element. In YAML, scope is designated by indentation. Where YAML builds upon RFC 822 and MIME for it's declaration section and support for binary and/or large scalar values, XML has it's own processing instruction mechanism and relies upon layered technologies for support for binary leaves. Furthermore, YAML has a simple whitespace policy, where XML's whitespace policy is completely configurable.
The terminology used to describe YAML is defined in the body of this specification. The terms defined in the following list are used in building those definitions and in describing the actions of a YAML processor:
A YAML document can reside in many different forms as long as it complies with the information model below. This includes a sequence of bytes in memory, on disk, or arriving via a network socket as much as it includes events from a sequential interface or an language specific in-memory representation. After covering the information model, this section focuses on the serialized representation
The information model for YAML is a directed acyclic graph having map, list, and scalar nodes. These data structures are directly supported in modern programming languages such as Python, Perl, Java, and C++.
Since the information model is a graph, a separate serialization model is required. The serialization model adds an additional node type, the reference, to record subsequent occurrences of a given node within a sequence. This enables a more compact notation so that duplicate occurrences of a given node need not be serialized more than once.
|An ordered sequence of anonymous map nodes.|
|A YAML node can be one of three types: list, scalar, map. Every node may optionally have a class.|
|An ordered sequence of zero or more nodes. Nodes are included by reference, thus they may be part of more than one list or map.|
|An ordered sequence of zero or more bits. Where bits are an atomic construct able to have a value of one or zero.|
|An un-ordered sequence of zero or more (key, node) tuples where the key is unique within the sequence.|
|A sequence of zero or more characters.|
|An opaque object that has a serialization which complies with the class production.|
|The serialization model adds an anchor attribute to every node, and introduces the reference node. The reference node advertises an anchor to indicate the repetition of a node previously encountered.|
|An additional attribute added to each node that provides for identification of the node within a given node sequence. Only nodes which could be referenced further in the sequence must be given an anchor. It is not necessary that the anchor is unique.|
|An additional node type consisting of an anchor which is used to indicate the repetition of the node with the same anchor most previously encountered.|
In the serialization model, it is important that each node is serialized exactly once. If a node appears more than once in the graph, only the first occurrence of the node should be serialized. All remaining occurrences of this node should be represented with reference nodes. In this scheme, if a YAML document is loaded into an random access representation, then the reference nodes and anchor indicators should not be available as the non-serialized information model should be used. Also note that anchors can repeat to allow for concatenation, although only the most recent node with a given anchor may be referenced.
Characters are the basis for a serialized version of a YAML document. Below is a general definition of a character followed by several characters which have specific meaning in particular contexts.
||::=||/* A single Unicode character representable by a 16 bit value, including the space and new line characters */|
Due to the end of line normalization
rules, the carriage return (#xD) is not included. As with
standard practice, the the surrogate block,
FFFF are excluded. Further, the range
[#x10000-#x10FFFF] is excluded since it would
require compliant YAML processors to be concerned with
surrogate pair handling.
In the information model scalar values may be binary and may not comply within the above character set. In this case, the scalar must be serialized using MIME section, perhaps using a base64 transfer encoding.
A YAML processor is required to support both UTF-16 and UTF-8 character encodings. If an input stream begins with a byte order mark, then the initial character encoding shall be UTF-16. Otherwise, the initial encoding shall be UTF-8.
||::=||/* The Unicode ZERO WIDTH NON-BREAKING SPACE character used to mark a UTF-16 stream and determine byte ordering. */|
If the stream begins with
the byte order of the input stream must be swapped
On input and before parsing, a compliant YAML parser must translate both the two-character sequence #xD #xA (CR LF) and any #xD (CR) which is not followed by a #xA (LF) into a single #xA (LF) character. This allows for the definition of an end-of-line marker.
||::=||/* a normalized end of line marker */|
||::=||/* a blank line */|
On output, a YAML emitter is free to serialize end of line markers using what ever convention is most appropriate. For Internet mail, CRLF is the preferred form.
Indicators are special characters which are used to describe the structure of a YAML document.
||::=||/* indicates a map node */|
||::=||/* indicates a list node */|
||::=||/* indicates a quoted string */|
||::=||/* separates a key from a node */|
||::=||/* indicates a block scalar */|
||::=||/* indicates end of a block */|
||::=||/* indicates an escape sequence */|
||::=||/* indicates a reference node */|
||::=||/* indicates a class attribute */|
||::=||/* indicates a null node */|
||::=||/* indicates an anchor attribute */|
||::=||/* indicates begin of binary scalar */|
||::=||/* indicates end of binary scalar */|
||::=||/* reserved */|
||::=||/* indicator characters */|
/* escape |
/* quote |
/* quote |
||::=||/* new line */|
||::=||/* tab */|
||::=||/* space */|
||::=||/* 8 bit character */|
||::=||/* 16 bit character */|
||::=||/* escape sequences */|
This section includes several common character range definitions.
||::=||/* Linear whitespace, the space or tab character. */|
||::=||/* A single whitespace character, including the space, tab, new line. Excluding the carriage return. */|
||::=||/* Non-whitespace characters */|
||::=||/* printables and linear whitespace */|
||::=||/* printables less the quote and escape character. */|
||::=||/* printables less the indicator characters. */|
||::=||/* ASCII alphabetic character, a-z and A-Z. */|
||::=||/* ASCII numeric character, 0-9 */|
||::=||/* ASCII alpha numeric character */|
/* one hexadecimal digit 0-9 |
/* chars for base64 |
Moving on to a higher level of abstraction, are sequences of characters, or strings. This section describes whitespace folding and indentation policies, as well as quoted and raw strings.
In a YAML serialization, structure is determined from indentation, where indentation is defined as an end of line marker followed by zero or more spaces or tabs. Indentation level is defined recursively.
||::=||/* The first level of indentation is a single end of line marker (new line). */|
||::=||/* The previous indentation setting plus one or more spaces or tabs. */|
It should be noted that this production does not imply that tab(3) is a constant thought the entire serialization, as the particular indentation style may change over time. Specifically, an indentation level tab(n) is set with the first subordinate line of a node at tab(n-1). With the exception of quoted strings, all subordinate lines must share this same indentation setting. This allows indentation to vary over the serialization as long as it remains constant for a given node's content.
Since a YAML serialization uses whitespace for indentation, in many cases the parser must condense sequences of one or more whitespace characters into a single space (#x20). When this functionality is implied, the fold production below will be used.
||::=||/* folded whitespace */|
/* linear folded |
With these definitions, there are two sorts of folded strings which are useful to describe.
/* folded |
/* indicator free, folded |
A folded string starts with a printable which is not an indicator. It is then followed by a sequence of printable words with intermediate folded whitespace. At parse time, the folded space must be condensed into a single space (#x20) character.
A quoted string is a mechanism to treat a sequence of characters as a single unit. Within a quoted string, indicators (with the exception of " and \) can be used without worry, indentation rules are suspended, and escape sequences can be used to introduce significant whitespace which would otherwise be subject to folding.
A quoted string beings and ends with a quote character. It can extend for as many lines as necessary, although an editor or emitter is free to re-break and indent a quoted string as needed to maintain readability. A quoted string cannot contain an un-escaped quote or an invalid escape sequence.
/* quoted |
An anchor string is a sequence of numeric digits used when referencing a node previously visited in the document stream.
||::=||/* Any sequence of digits 0-9. */|
Each node can be adorned with a class string. There are three types of class strings: local, qualified, and built-in.
||::=||/* A local, qualified or built-in class name. */|
Local class strings, which are unique within a controlled context, are a sequence of printable characters without save the period. These strings may not be globally unique and should only be used for local usage or for experimentation where data exchange is not likely.
||::=||/* A local class string */|
Qualified class strings are globally unique since they are serialized with a reverse domain name format similar to "com.clarkevans.timesheet". The definition of a qualified class is determined by the domain name holder.
||::=||/* A reverse dns qualified class string */|
Built-in class strings, reserved for YAML usage, are also globally unique and begin with a period. Core classes include real, designating an inprecise floating point value, and int, designating a precise integer value.
||::=||/* A built-in class name */|
The above definitions depend upon domain name strings. These are defined below.
||::=||/* A reverse dns name */|
||::=||/* A top level domain, like "com", "org", ... */|
||::=||/* A domain name segment */|
A serialized object is a YAML document if, taken
as a whole, it complies with the following production.
Since the YAML serialization depends upon
indentation level to delineate blocks, each
node production is a function of an integer,
much like the
production above. This production is the
beginning of the recursion.
/* A byte order mark followed by a sequence of
maps seperated by a blank
||::=||/* an optional node color followed by either a map, list, scalar, or reference */|
/* either an anchor, or class, or both in any |
YAML indirectly supports the serialization of data types and object class name with a class attribute on each node. If this attribute is provided, then each specific language binding may use this information to dress the node appropriately, otherwise a warning must be issued and the must data treated as if the class attribute was not provided.
||::=||/* Associates an class string with a given node */|
built-in: #.real 23.42 local: #local Local class name. non-class: Free of class. qualified: #com.clarkevans.ts Qualified class name.
An anchor is an indicator which can be used to color a node giving it an sequential numeric digit for an identifier. The reference node type can then be used to indicate additional inclusions of an anchored node. The anchor string of a reference refers to the most recent node having that same anchor string. It is an error to have a reference use an anchor string which does not occur previously in the serialization.
||::=||/* Associates an anchor string with a given node for further reference */|
||::=||/* A reference node */|
anchor: &0001 This scalar has an anchor. clsanc: #cls &0002 Has alocal class and an anchor. non-ref: Next node refers to the previous. reference: *0002
A list is the simplest form of node, it is a sequence of nodes at a higher indentation.
/* A list of zero or more indented |
list: @ First item in top list @ Subordinate list @ Above list is empty Fifth item in top list
A map is a unique association of keys with values. Where a key is either a quoted or folded string.
||::=||/* A map indicator, followed by a list of map items */|
/* A key/node map pair indented appropriately.
Either a space, or the appropriate indentation
||::=||/* a single line folded string, or a quoted string */|
||::=||/* node, including the multiline, with map specific whitespace handling options. */|
In a given map, there is the further restriction that within a given map, two folded key values cannot be identical. Also, the canonical serialization orders the keys alphabetically.
map: % first : First entry second: % key: Subordinate map! third item: @ Subordinate list % Previous map is empty. "@": This key had to be quoted.
The while most of the document productions above are fairly strict, the scalar production is generous. It offers four in-line styles of expressing scalar values depending upon the readability requirements. The table below describes the various styles.
The following scalar forms can be used in lists, blocks, and as part of the top level sequence
||::=||/* in-line scalar nodes */|
For this case, each line is indented, whitespace is folded, escaping is not available, and the content must not start with an indicator. To delineate the end of this scalar, indention is employed.
/* one or more indented, non-escaped,
first: This is a one line scalar. second: <html><head><title>Embedded HTML!</title></head> <body><p class="none">This can even have embedded HTML since there is no escaping, and since < (the starting character) is not an indicator! </p></body> third: Indicators like @ : % are allowed, as well as quotes, as long as the first character is not an indicator. Further, whitespace is folded.
This mechanism supports source code or other scalar values with significant use of indicators, quoted escaping, or significant whitespace. To retain readability for this case, the block scalar allows raw text strings to be used in an indented fashion where each line indented is preceded by a block indicator. All whitespace is significant up to and including the end of line marker. To end the block, a block end indicator is used and any characters on this line are also included, up to and not including the end of line marker.
first: |This is a block scalar, with significant |whitespace, and the use of " @, etc. |The \ is used to signify the end of the block. | Whitespace *is* significant. \ second: \No leading nor trailing new line. third: | |First list item which has a |leading and trailing new line. \ |Second list item. Does not have |leading nor trailing new line. |Does have three new lines in the \middle.
A quoted scalar uses quoted strings. With this mechanism, the indicators (excepting the quote and slash) can be used freely, whitespace is folded, and escape sequences can be used to indicate significant whitespace. Since a quoted string has an end marker, indenting rules are suspended for its content allowing for easy cut and paste, although quotes, significant whitespace, and slashes must still be escaped.
||::=||/* A quoted scalar value. */|
first: "Quoted scalar.\nWith a new line." second: @ "Spaces are folded so this new line is not significant." "Furthermore indicators such as @ # : can be added." "Escape sequences can be used to removed or \ add white\ space, \\ and \" must be escaped." "These quote lines may also break indentation." "This was a list of five quoted scalars!"
A binary scalar uses the base64 encoding. Base64 strings are a sequence of base64 characters intermixed with whitespace which is ignored by the base64 processor. An additional constraint is that no more than 76 sequential characters are allowed without at least one intermediate whitespace character.
/* A base64 encoded binary |
binary: [R0lGODlhDAAMAIQAAP//9/X17unp5WZmZgAAAO fn515eXvPz7Y6OjuDg4J+fn5OTk6enp56enmlp aWNjY6Ojo4SEhP/++f/++f/++f/++f/++f/++f /++f/++f/++f/++f/++f/++f/++f/++SH+Dk1h ZGUgd2l0aCBHSU1QACwAAAAADAAMAAAFLCAgjo EwnuNAFOhpEMTRiggcz4BNJHrv/zCFcLiwMWYN G84BwwEeECcgggoBADs=] desc: The binary value above is a tiny arrow encoded as a gif image.
There are two basic types of interfaces used for sequential processing, pull and push. The difference lies primarly in who has the primary control of the process, the consumer or the producer. With a push interface, the producer has the primary control and with a pull interface is is the consumer who drives the process. For the most control over the process, YAML's parser has a pull interface and YAML's emitter has a push interface. This enables the application which pulls from the parser and pushes to the emitter to have the primary control of the process.
The parser or iterator interface provides a sequential access to a YAML document where the flow control is dicated by the consumer. The parser interface uses the serialization model, handling duplicate nodes within the graph through a reference. The parser interface has several components.
|A structure is used to hold the current node, including the node's type (scalar, list, map, or reference), the node's class, the node's anchor, and, if the node is part of a map, the key associated with the node.|
|A function which returns the document's top level list cursor. This method may require an input stream such as a string buffer, a file object, or a open socket.|
|A function which takes a map or list cursor and returns a cursor for the first child within the sequence, if any. It is an error to call this function on cursor currently holding a reference or scalar.|
|A method which advances the cursor passed to the next node within the current map or list. If there is not a subsequent node within the sequence, then this function must free the cursor and should indicate that additional siblings are not to be found.|
|A method on the top level cursor which can be used on the top level cursor to stop iteration and invalidate all subordinate cursors.|
|A method which can be called on a scalar cursor to read an arbitrary sized chunk of it's value. A scalar's value can only be read once, but multiple calls to this function may be required.|
|An optional method which can be called on a cursor with a reference node to return a new cursor for the node referenced. This method need only be supported by those iterators over a random access input stream.|
As part of this interface, each cursor returned from the parser must remain valid until close() is called, or the next() is called on the cursor, or next() is called on an ancestral cursor. Note that this implies that N cursors may be open at any given time, one cursor for each map or list in a given lineage. It is the parser's task to make sure that each cursor points to valid information.
The emitter or visitor interface interface provides a sequential access to a YAML document where the flow control is dicated by the producer. The emitter interface uses the serialization model, handling duplicate nodes within the graph through a reference. The emitter interface has several components.
|The emitter may require a cursor for its bookkeeping which is opaque.|
|A function which must be called to start the emitter. This function returns a cursor to be used for subsequent operations.|
|A function to be called when a map, list, or scalar node is to be started. This function is passed a the current cursor and returns a cursor to be used for subordinate operations. This cursor takes the node's type, a class, an anchor, and possibly a key if the node is in the context of a map.|
|A method which can be called multiple times within the context of a scalar to provide the scalar's character value.|
|An optional method which can be called on a map or list cursor to add a reference to the sequence of children. This method takes the current cursor and an anchor as parameters. If this method is not provided, the caller must recursively visit the referenced node.|
|A method called on a cursor to indicate that the node in question is no longer in scope. The cursor passed should no longer be used.|
|A method which must be called on the top level cursor to indicate that the document stream has finished.|
As part of this interface, a cursor is deemed valid untill end() has been called on it. It is an error to call end() on a cursor when a subordinate cursor is still valid. This implies that the emitter need not use N cursors, and that one cursor which is re-used with a depth indicator will suffice.
In some cases it is necessary to convert a pull to a push interface, or a push to a pull interface. While converting from pull to push is relatively easy, converting from push to pull requires a multi-threaded approach or a large intermediate memory buffer. To help automate conversions, the YAML processing library includes both interface converters.
The realization of the information model depends upon the programming language. Included in this specification are the official bindings for Python, Perl, and C. A language binding is not required to implement the sequential interface.
TODO: This section needs much work
Since the "C" programming language does not have dynamic lists or maps, this binding only implements the sequential interface.
For the Python binding, unless specified otherwise by a class, scalar values are bound to the string type, lists are bound to the list type, and maps are bound to the dictionary type with the key represented as a string.
If a scalar has a real class, then the scalar must be bound as a floating point value if possible, otherwise a warning must be issued. Likewise, if a scalar has a int class, then the scalar must be bound to a normal integer, otherwise a warning must be issued. Conversions can fail if the scalar value is not parse-able or if the size of the parsed value will not fit within the standard type.
If any other class is encountered, then the results depend upon the type. First, the class identifier is used to construct a class with the same package name. And then, depending upon the type, initialization is carried out differently. If the node is a map, then the keys are assumed to be attributes, and the setattr method is used to initialize the object. If the node is a scalar, then repr is used. Otherwise, if the node is a array, then additem method is used.
For the Perl binding...
For the Java binding...