# Wrapping a C library with a streaming interface

I've tried to give this a generic title to help future searchers, but concretely I am trying to parse YAML data. Import doesn't seem to support it natively and I can't find any code for it online, so I thought the easiest approach would be to wrap a C library (libyaml).

I haven't used the WSTP stuff yet, but the documentation looks pretty good at least for the easy cases, I was wondering if someone give give some tips though on

• To send an association using MathLink, just send an expression such as Association[Rule["foo", 1], Rule["bar", 2]]. There's nothing special about associations compared to other expressions when sending them through MathLink. Can you explain about streaming APIs to those of us who are not familiar with this concept? What difficulty do you expect and why? – Szabolcs Sep 27 '16 at 10:41
• If you mean that you read the parts one by one, you can also just keep writing the result to the (Math)Link one by one, as you receive them. However, it is necessary to know when there are no more items, so you can wrap up things. – Szabolcs Sep 27 '16 at 10:50
• How's this project going? – Szabolcs Sep 29 '16 at 11:00
• Another YAML importer was just published. You can still post yours as an answer. mathematica.stackexchange.com/a/128506/12 – Szabolcs Oct 12 '16 at 6:44
• @Dan haha, the odds that we both write a YAML importer on SnakeYAML within 48 hrs of each other... :) Tip: calling into Java is expensive, so doing the parsing from Mathematica is slower than doing it in Java. For a 30kb document, for example, yours takes 0.6 seconds; vs. 0.02 sec done in Java. Compare: github.com/zbjornson/MYaml/blob/master/javaSource/… – ZachB Oct 12 '16 at 17:26

You can use MathLink (recently called WSTP) for this.

MathLink is an interprocess communication protocol. You can create links and send arbitrary Mathematica expressions through it. This is the must-read tutorial for MathLink. Use it in conjunction with the documentation.

Typically, you would create a function that is callable directly from Mathematica, e.g. by creating an installable MathLink program. You will need to write the expression to be returned by this function to a MathLink link. When using Installable programs, you simple set the return type to Manual in your template file and write to stdlink.

When parsing a YAML file, you probably need to anticipate errors and maybe just return $Failed if an error has occurred. To deal with this I suggest not writing the expression to stdlink at first. Instead, create a loopback link. Write the parts of the expression to the link one by one, as you receive them from libyaml. When you're finished, just transfer everything to stdlink. If you must abort because of the error, throw away the contents of the loopback link, and MLPutSymbol a $Failed symbol in stdlink.

Sending associations is the same as sending any other expression. Send them in the form Association[Rule["a", 1], Rule["b", 2]], etc.