Services Case Studies Blog About Contact
Phone Github Twitter Linkedin Email

How Compliant Is Psych? How Much Better Is libfyaml?

Psych offers a libfyaml backend now. Do you need it?

· 21 minute read
Share:
Share on Facebook Share on Twitter Share on LinkedIn Share by Email

YAML has seen some changes over the years - even it’s name has shifted from “Yet Another Markup Language” to “YAML Ain’t Markup Language.” Unsurprisingly, so have YAML parsers. Rubyists, for example, used to use the syck gem, which was based on the syck library; that was replaced by Psych, which was originally based on libyaml.

On August 20, 2026, though, Psych added support for a different backend, libfyaml, noting that it has strict YAML 1.2 compatibility - which raises several questions: first, how different is YAML 1.1 and 1.2? How bad is the Psych YAML 1.2 compatibility? Is the libfyaml backend for psych that much better?

In order to answer that question, I’ve setup a test harness for Psych - running it against the YAML conformance suite and comparing it with several other YAML parsers.

Note that we’re going to be focusing on syntax here - there’s an entirely different can of worms involving how the values are interpreted, and that may be a subject of a future value.

You can peruse the source code and run it yourself at GitHub:

https://github.com/djberube/djb-yaml-test-harness

In my testing, Psych passes 315 of 405 (77.8%) of syntax test cases for YAML. The new Psych new libfyaml backend passes 403/405 (99.5%) - which is quite a bit higher. Does it matter?

First, this shouldn’t be read as “Psych with libyaml bindings can read only 77.8% of YAML documents”; spec suites cover a lot of cases, many of which are quite contrived by design.

On the other hand, as noted above, that 77.8% number only convers syntax conformance - just checking that a given input parses to a particular set of event streams. Casting values to actual host language types falls outside of that purview; we will touch on it briefly for educational purposes, but the test suite does not cover that part of the process..

In any event, we’re going to be covering some very specific cases where Psych doesn’t fit the spec - so that in the very specific situation that you get bit by one of these incompatibilities, you will be able to recognize it and remedy the situation.

The idea here is to help you become familiar with what differences Ruby’s libyaml bindings have with what happens in other languages; at Durable Programming, we often have to function in polyglot environments, and this is crucial knowledge when debugging complex interchange scenarios.

Specifically, we are going to be comparing our Ruby parse output with a few other parsers from other languages - twelve total parsers, in fact.

Here’s the full list:

Notably, the two libyaml implementations behave quite similarly, as we’ll see; also note that the Python safeyaml implementation was written by the author of libyaml, Kirill Simonov, so it’s reasonable to expect some similar behaviours to the libyaml implementations.

One final note before we get started: Psych only claims to be YAML 1.1 compatible. However, some of its behaviour is 1.2 compatible but not 1.1 compatible. If this article judged it byb 1.1 standards only, some might argue that this is unfair, since YAML 1.2 adjusts the spec to match what some parsers in the ecosystem already did. Conversely, if we judged only by YAML 1.2 conformance, some readers might argue that Psych never directly claims to be 1.2 compatible - the README and the RDoc never directly say either way.

The 1.2 spec itself notes the following: “Note that version 1.2 is mostly a superset of version 1.1, defined for the purpose of ensuring JSON compatibility.” At least in theory, 1.1 valid syntax should mostly be valid 1.2 sytnax.

Therefore, we will try to note both behaviours and let you make your own decision.

Directive support

Let’s start our look at YAML parsing at the very top - YAML directives. If a file has them, they are going to be at the beginning. A YAML directive contains meta-information about the document - as the YAML 1.2 spec says, “[d]irectives are instructions to the YAML processor and … are not reflected in the YAML serialization tree”; they start with a %, and the two defined by the YAML spec are TAG and YAML.

Of those two, the one you are most likely to be familar with is %YAML:

%YAML 1.1
---
a: 3

… or perhaps like this:

%YAML 1.2
---
b: 3

You can think of YAML directives like <!DOCTYPE html> - they help the parser realize what version of the document to expect. Unlike DOCTYPE tags, which should always be included, YAML directives are an optional feature.

As to the examples above, Psych can parse both of them just fine:

YAML.load("
%YAML 1.1
---
a: 3")
# =>
# {"a" => 3}

YAML.load("
%YAML 1.2
---
b: 3")
# =>
# {"b" => 3}

Of course, %YAML isn’t the only directive available - there’s also the %TAG directive, which, though its not frequently used, is also supported by Psych, which is nice.

Interestingly, the YAML spec says parsers “should” simply pass through any directives they don’t understand; specifically, YAML 1.2.2 §6.8.3: “reserved directives … should be ignored with a warning.”

Likewise, YAML 1.1 §7.1 says this about unknown directives: “This specification defines two directives, ‘YAML’ and ‘TAG’, and reserves all other directives for future use … A YAML processor should ignore unknown directives with an appropriate warning.”

Psych does not implement this behaviour; instead, it throws an error:


YAML.load("%UNKNOWNDIRECTIVE
---
x: 3
") 
# =>
# ERROR: Psych::SyntaxError: (<unknown>): found unknown directive name while scanning a directive at line 1 column 1

Let’s see what Python does:

import yaml

yaml.safe_load("%UNKNOWNDIRECTIVE \n---\nx: 3")  

# =>
# {'x': 3}

No error - it correctly disregards the %UNKNOWNDIRECTIVE… although it doesn’t emit a warning either, which means it’s spec-noncompliant in a different way.

Now, Python’s PyYAML has two parsers, one written in Python, which we saw above, and one which is a binding to libyaml; lets try the libyaml one.

import yaml

yaml.load("%UNKNOWNDIRECTIVE \n---\nx: 3", Loader=yaml.CSafeLoader)

# =>
# ERROR: ScannerError: while scanning a directive

Both Psych and PyYAML libyaml binding throw errors because of underlying choices in libyaml here, which is worth noting.

Lets try this with Psych backed by libfyaml:

require 'yaml'

YAML.load("%UNKNOWNDIRECTIVE \n---\nx: 3", aliases: true)
# =>
# {"x" => 3}

Unlike libyaml, the fyaml binding correctly parses the document.

Similarly, rapidyaml targets YAML 1.2, and can handle unknown directives:

from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("%UNKNOWNDIRECTIVE \n---\nx: 3")
# =>
# {'x': 3}

As with Python’s PyYAML native parser and with fyaml, it ignores with no warning - not quite up to spec, but at least its not rejecting otherwise valid documents. (Some would argue that over-rejecting is better than over-accepting; I’ll leave that to your own judgement.)

Let’s take a look at the js-yaml package for nodejs:

const yaml = require('js-yaml');

yaml.load("%UNKNOWNDIRECTIVE\n---\nx: 3\n"); 
// =>
// { x: 3 }

Likewise, no error here.

In total, 9 out of the 12 parsers in our test suite accept %UNKNOWNDIRECTIVE - it’s just Psych, PyYAML safeyaml, and go-yaml that reject it.

Also, the YAML spec says that the parser should throw an error when it receives a YAML directive for a higher minor version than it supports. Instead, Psych just ignores it and continues, possibly misparsing the document:

require 'yaml'

YAML.load("%YAML 1.2\n---\na: 3")
# =>
# {"a" => 3}

The fyaml backend does the same:

require 'yaml'

YAML.load("%YAML 1.2\n---\na: 3", aliases: true)
# =>
# {"a" => 3}

Next, note that the 1.1 and 1.2 both specs both say higher major version numbers should be outright rejected with an error message; the 1.1 spec section 7.1 and the 1.2 spec section 6.8.1 both read identically, as follows: “Documents with a “YAML” directive specifying a higher major version (e.g. “%YAML 2.0”) should be rejected with an appropriate error message.”

Psych does, as it so happens, do this correctly, both with the libyaml and fyaml bindings:

require 'yaml'

YAML.load("%YAML 2.0\n---\na: 3")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): found incompatible YAML document at line 1 column 1
require 'yaml'

YAML.load("%YAML 2.0\n---\na: 3", aliases: true)
# =>
# ERROR: [ERR]: unsupport version number 2.0
# Psych::SyntaxError: (<unknown>): could not parse YAML at line 0 column 0

In fact, libyaml is in the majority here - of our twelve parsers, seven threw an error and five ignored it.

Both specs suggest multiple %YAML tags should result in an error:

require 'yaml'

YAML.load("%YAML 1.1\n%YAML 1.1\n---\na: 3")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): found duplicate %YAML directive at line 1 column 1

… which they do.

While nonstandard directives are terribly common, they aren’t unheard of, either; quite old versions of the OpenCV computer imaging framework has documents like this:

%YAML:1.0
---
calibration_time: "2026-08-31 10:15:00"
image_width: 1920
image_height: 1080
board_width: 9
board_height: 6
square_size: 25.0
camera_matrix: !!opencv-matrix
   rows: 3
   cols: 3
   dt: d
   data: [ 1.4200000000000000e+03, 0., 9.6000000000000000e+02,
           0., 1.4200000000000000e+03, 5.4000000000000000e+02,
           0., 0., 1. ]

That first line - the %YAML:1.0 directive, with a colon instead of a space - causes Psych to raise an error, as we see here:

require 'yaml'

YAML.load("%YAML:1.0\n---\ncalibration_time: \"2026-08-31 10:15:00")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): found unexpected non-alphabetical character while scanning a directive at line 1 column 1

There are other situations where custom directives might be seen; a quick search on Github Code Search shows “%SCHEMA”, “%include”, and more.

In short, Psych doesn’t have a consistent position here; it blows up on any unknown directive, which isn’t required or even mentioned as an option by the spec, but it doesn’t blow up on the one thing that the spec requires it to do: throw a warning on a version of YAML that it doesn’t support. There’s not a clear reason why multiple %YAML directives error but a single %YAML directive with an incorrect major version does not.

Scalars, Special Characters, and Flow Context

YAML has a surprisingly flexible syntax; you may be surprised to learn that YAML 1.2 has a goal of supporting all JSON syntax, for example. Indeed, even pre-1.1 YAML has support for multiple “styles”: flow style and block style. The examples so far have all been block style; this is likely the most familiar form of YAML to many of you.


# Block style
YAML.load("x: 3") 
# => 
# {"x" => 3}

# Block style with a start of document marker
YAML.load("---\nx: 3") 
# => 
# {"x" => 3}

# Flow style

YAML.load("{x: 3}") 
# => 
# {"x" => 3}

The above are all valid YAML syntax; the first two are block style, and the last one is flow style. They keys used here are bare names; as we can see in the output, Psych converts them to strings, which is reasonable. We can also specify them as strings by writing "x" instead of x - in the above case, it functions just the same… but that is not always the case, and Psych had some odd behaviour relating to this.

Now, not all keys are alphanumeric. In fact, YAML 1.1 §9.1.3 specficially spells out the fact that either a key or a value should be able to start with a :, a ?, or a -; likewise, YAML 1.2.2 §7.3.3 permits : to start a flow-context plain scalar when its not followed by a space.

Does Ruby’s psych binding allow such?


YAML.load("{x: :x}") 
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected node content while parsing a flow node at line 1 column 5

It appears that Psych is interpreting the : in :x here as a grammatical marker and not a part of the value, despite what the standard indicates.

Let’s see what Python does:

import yaml

yaml.safe_load("{x: :x}") 
# =>
# ERROR: ParserError: while parsing a flow node

Let’s try Psych’s fyaml binding binding:

require 'yaml'

YAML.load("{x: :x}", aliases: true)
# =>
# {"x" => :x}

Let’s take a look at the js-yaml package for nodejs:

const yaml = require('js-yaml');

yaml.load("{x: :x}"); 
// =>
// { x: ':x' }

rapidyaml reads it the same way js-yaml does:

from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("{x: :x}")
# =>
# {'x': ':x'}

So while this isn’t strictly a libyaml issue - the safeyaml Python parser behaves the same as libyaml psych - there are a number of other implementations that handle this correctly.

As we will discuss more shortly, note that psych does /not/ return ":x" for the value when used with the fyaml binding - it converts it to a symbol. Because the : gets removed, note that ":x" != :x - the symbols name is x, not :x.

Somewhat counter-intuitively, libyaml-based Psych does support values starting with a : in a non-flow context:


YAML.load("---
x: :x") 
# =>
# {"x" => :x}

Notice that, as before :x gets turned into a symbol; this is a convenience provided by Psych. The 1.1 spec doesn’t specifically mention a string vs symbol distinction; this is a Ruby language issue, but its mentioned here as it may very well become a real interoperability issue if this behaviour results in your code comparing strings with symbols. Regardless of your opinion on this specific Ruby quirk, the auto-symbolizing behaviour - and the above error - are something you should know.

Interestingly, Psych lets us have a scalar starting with a : in a flow context if you add quotes around the scalar:


YAML.load("---
x: ':x'") 
# =>
# {"x" => ":x"}

… except that, as you likely noticed, we got ":x", not :x - Psych does not automatically convert a quoted scalar starting with : into a symbol.

One might wonder - is there a way to force Psych to symbolize this without adding special cases to your code? Now, Psych does have a symbolize_names: option, but it doesn’t help here, since it only symbolizes keys:


YAML.load("---
x: ':x'", symbolize_names: true) 
# =>
# {x: ":x"}

The behaviour of this option is non-obvious: symbolize_names: false does not prevent keys prefixed with a bare : from being turned into symbols:


YAML.load("---
:x: ':x'", symbolize_names: false) 
# =>
# {x: ":x"}

The above code is, quite literally, psych symbolizing a name when symbolize_names is false.

Note that neither for symbolize_names will convert an explicitly quoted key like ':x' into a symbol:


YAML.load("---
':x': ':x'", symbolize_names: true) 
# =>
# {":x": ":x"}

YAML.load("---
':x': ':x'", symbolize_names: false) 
# =>
# {":x" => ":x"}

Since the symbolization happens inside the Ruby code - not in libyaml, it’s not surprising that Psych with libfyaml works much the same:


YAML.load("---
':x': ':x'", symbolize_names: true) 
# =>
# {":x": ":x"}

YAML.load("---
':x': ':x'", symbolize_names: false) 
# =>
# {":x" => ":x"}

The conclusion, therefore, is that symbolize_names only controls the conversion of bare scalar keys not prefixed with : - which, frankly, is not a very intuitive result.

Briefly, note that this behaviour applies to non-colon special characters, such as { or } - though realistically you’ll likely be quoting values like "{{2" anyway, and it’s only the special casing that Psych does for symbols that makes the case of : jump out.

Finally, one thing that Psych does correctly is scalars that consist only of a special character, like this:

YAML.load("x: :")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): mapping values are not allowed in this context at line 1 column 4

The YAML 1.1 spec states that this is only valid if the special character is followed by a non-special character - so the above behaviour is, in fact, correct.

Inconvenient Line Breaks

The YAML 1.1 spec allows a linebreak in flow context after the key but before the colon; Psych does not allow this.

require 'yaml'

YAML.load("{a\n: 3}")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected ',' or '}' while parsing a flow mapping at line 1 column 1

Python chokes on it as well:

import yaml

yaml.safe_load("{a\n: 3}")
# =>
# ERROR: ParserError: while parsing a flow mapping

… though node handles it just fine:

const yaml = require('js-yaml');

yaml.load("{a\n: 3}");
// =>
// { a: 3 }

… as does rapidyaml:

from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("{a\n: 3}")
# =>
# {'a': 3}

After the colon, however, is handled by Psych just fine:

require 'yaml'

YAML.load("{a: \n3}")
# =>
# {"a" => 3}

Multiline strings do not work for keys:

require 'yaml'

YAML.load("{\"line\ntwo\": \n3}")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected ',' or '}' while parsing a flow mapping at line 1 column 1

… though note that’s not a spec compliance issue; YAML 1.2 requires it, but YAML 1.1 does not.

Which gives us a way to check that claim rather than assert it. rapidyaml is a 1.2 parser, so if the reading above is right, it should accept the document:

from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("{\"line\ntwo\": \n3}")
# =>
# {'line two': 3}

It does, and it folds the break into a space, which is what 1.2 asks for.

Anchoring Monstrosities

Anchors are a nifty feature of YAML, and help you DRY up complex configs.

Perhaps you have seen a Rails configuration like this:

default: &default
  adapter: postgresql
  encoding: unicode
  host: <%= ENV.fetch("DATABASE_HOST") { "localhost" } %>
  username: <%= ENV.fetch("DATABASE_USERNAME") { "postgres" } %>
  password: <%= ENV.fetch("DATABASE_PASSWORD") { "" } %>
  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
  timeout: 5000

development:
  <<: *default
  database: myapp_development

test:
  <<: *default
  database: myapp_test

# ... etc ...

The above code interpolates the default anchor into the three environments, with each one adding a bit of data. This is how you usually see anchors used, and it’s quite reasonable.

However, according to a strict reading of the YAML 1.1 spec, an anchor is a & followed by any number of nonspace characters. This means that a: is a valid anchor name - even if it’s an unnecessarily confusing one.

However, neither Psych nor Python much like this correct-but-confusing reading:

require 'yaml'

YAML.load("&a: key: value\nb: *a")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): mapping values are not allowed in this context at line 1 column 8
import yaml

yaml.safe_load("&a: key: value\nb: *a")
# =>
# ERROR: ScannerError: mapping values are not allowed here

… although NodeJS, RapidYaml, and Psych fyaml are quite fine with it:

const yaml = require('js-yaml');

yaml.load("&a: key: value\nb: *a:");
// =>
// { key: 'value', b: 'key' }
from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("&a: key: value\nb: *a:")
# =>
# {'key': 'value', 'b': 'key'}
require 'yaml'

YAML.load("&a: key: value\nb: *a:", aliases: true)
# =>
# {"key" => "value", "b" => "key"}

You can go unnecessarily far with this. As I said above, the strict reading of the spec is any printable nonspace character - meaning that, technically, emojis should be fair game.

As before, Psych libyaml and Python don’t much care for this one:

require 'yaml'

YAML.load("&\u{1F48E} key: value\nb: *\u{1F48E}")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected alphabetic or numeric character while scanning an anchor at line 1 column 1
import yaml

yaml.safe_load("&\U0001F48E key: value\nb: *\U0001F48E")
# =>
# ERROR: ScannerError: while scanning an anchor

.. but NodeJS, RapidYAML, and Psych FYAML are both OK with it:

const yaml = require('js-yaml');

yaml.load("&\u{1F48E} key: value\nb: *\u{1F48E}");
// =>
// { key: 'value', b: 'key' }
from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("&\U0001F48E key: value\nb: *\U0001F48E")
# =>
# {'key': 'value', 'b': 'key'}
require 'yaml'

YAML.load("&\u{1F48E} key: value\nb: *\u{1F48E}", aliases: true)
# =>
# {"key" => "value", "b" => "key"}

Technically, the spec lets you have just : as an anchor name; both LibYAML bindings fail on this, as does safeyaml:

require 'yaml'

YAML.load("&: key: value\nb: *:")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected alphabetic or numeric character while scanning an anchor at line 1 column 1
import yaml

yaml.load("&: key: value\nb: *:", Loader=yaml.CSafeLoader)
# =>
# ERROR: ScannerError: while scanning an anchor
import yaml

yaml.safe_load("&: key: value\nb: *:")
# =>
# ERROR: ScannerError: while scanning an anchor

… but our NodeJS, RapidYAML, and Psych libfyaml bindings all work:

const yaml = require('js-yaml');

yaml.load("&: key: value\nb: *:");
// =>
// { key: 'value', b: 'key' }
from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("&: key: value\nb: *:")
# =>
# {'key': 'value', 'b': 'key'}

require 'yaml'

YAML.load("&: key: value\nb: *:", aliases: true)
# =>
# {"key" => "value", "b" => "key"}

In French, “chargé d’affaires” means something like “account manager”; lets see if we can use chargé_d’affaires as a key:

require 'yaml'

YAML.load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected alphabetic or numeric character while scanning an anchor at line 1 column 1
import yaml

yaml.safe_load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires")
# =>
# ERROR: ScannerError: while scanning an anchor
import yaml

yaml.load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires", Loader=yaml.CSafeLoader)
# =>
# ERROR: ScannerError: while scanning an anchor
const yaml = require('js-yaml');

yaml.load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires");
// =>
// { key: 'value', b: 'key' }
from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires")
# =>
# {'key': 'value', 'b': 'key'}
require 'yaml'

YAML.load("&charg\u00E9_d\u2019affaires key: value\nb: *charg\u00E9_d\u2019affaires", aliases: true)
# =>
# {"key" => "value", "b" => "key"}

Both libyaml based parsers and the Python safeyaml parser fail here and the others pass.

Comments must be preceded by whitespace

Strictly speaking, comments should be preceded by whitespace; both of our libyaml based parsers, however, skip this requirement.

require 'yaml'

YAML.load("[3, 4, 5]#test")
# =>
# [3, 4, 5]
import yaml

yaml.safe_load("[3, 4, 5]#test ")
# =>
# [3, 4, 5]

Both of our non-libyaml based parsers enforce this requirement:

const yaml = require('js-yaml');

yaml.load("[3, 4, 5]#test");
// =>
// ERROR: YAMLException: end of the stream or a document separator is expected (1:10)
from core_schema import load  # YAML 1.2 core schema over ryml's tree

load("[3, 4, 5]#test")
# =>
# ERROR: ExceptionParse: comment not preceded by whitespace

For more information, check out examples 9JBA, CVW2, SU5Z, and X4QW in the YAML test suite and §6.2 in the 1.1 YAML spec.

Bonus: Starts and Ends

Likely, you’re familar with the YAML beginning of document marker ---. Perhaps a bit less familiar is the end of document marker, ... - the idea being if you’re streaming YAML, and there is some time between document transmissions, you can finish processing your document as soon as you get the ... - no need to wait for the next document’s --- marker.

Now, Psych does not allow a document consisting of just a ...:

require 'yaml'

YAML.load("...")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected node content while parsing a block node at line 1 column 1

… but a lone starting marker works just fine:

require 'yaml'

YAML.load("---")
# =>
# nil

Both markers together parse just fine:

require 'yaml'

YAML.load("---\n...")
# =>
# nil

However, a comment and then a closing marker blows up:

require 'yaml'

YAML.load("# test\n...")
# =>
# ERROR: Psych::SyntaxError: (<unknown>): did not find expected node content while parsing a block node at line 2 column 1

… but just a comment is OK:

require 'yaml'

YAML.load("# test")
# =>
# nil

Conclusion

None of the above means that Psych or libyaml is a bad library; much of the above is likely not relevant to you, and for the parts that are, it may be that Psych’s behaviour is better for you than the actual spec.

Nevertheless, there are differences, and it’s useful to be aware of them. You may not run into any of the above cases often - but if you do, familiarity with where Psych is spec-incompliant will likely help you diagnose.

libfyaml does perform better than libyaml on the conformance suite - but, as we saw above, there’s a decent chance the particular failures won’t impact your application, so whether or not its worth the effort to switch now is going to be a judgement call.