mirror of
https://github.com/ocaml-tracing/ocaml-trace.git
synced 2026-03-09 12:23:32 -04:00
102 lines
43 KiB
HTML
102 lines
43 KiB
HTML
<!DOCTYPE html>
|
||
<html xmlns="http://www.w3.org/1999/xhtml"><head><title>writing-ppxs (ppxlib.writing-ppxs)</title><meta charset="utf-8"/><link rel="stylesheet" href="../_odoc-theme/odoc.css"/><meta name="generator" content="odoc 3.1.0"/><meta name="viewport" content="width=device-width,initial-scale=1.0"/><script src="../highlight.pack.js"></script><script>hljs.initHighlightingOnLoad();</script></head><body class="odoc"><nav class="odoc-nav"><a href="index.html">Up</a> – <a href="../index.html">Index</a> » <a href="index.html">ppxlib</a> » writing-ppxs</nav><header class="odoc-preamble"><p> <div style="display: flex; justify-content:space-between"><div><a href="driver.html" title="driver">< The Driver</a> </div><div><a href="generating-code.html" title="generating-code">Generating AST nodes ></a> </div></div></p><h1 id="writing-a-transformation"><a href="#writing-a-transformation" class="anchor"></a>Writing a Transformation</h1><p>This chapter covers the <code>ppxlib</code> procedure basics to define and register a transformation, be it a global or a context-free transformation.</p><p>For the actual manipulation and generation of code, <code>ppxlib</code> provides many helpers that are listed in <a href="#generatingcode" title="generatingcode">Defining AST Transformations</a>.</p></header><div class="odoc-tocs"><nav class="odoc-toc odoc-local-toc"><ul><li><a href="#defining-a-transformation">Defining a Transformation</a></li><li><a href="#context-free-transformation">Context-Free Transformation</a><ul><li><a href="#extenders">Extenders</a><ul><li><a href="#ext_context">The Extender Context</a></li><li><a href="#the-extender-name">The Extender Name</a></li><li><a href="#the-payload-extraction">The Payload Extraction</a></li><li><a href="#the-expand-function">The Expand Function</a></li><li><a href="#declaring-an-extender">Declaring an Extender</a></li></ul></li><li><a href="#derivers">Derivers</a><ul><li><a href="#derivers-arguments">Derivers Arguments</a></li><li><a href="#derivers-dependency">Derivers Dependency</a></li><li><a href="#generator-function">Generator Function</a></li><li><a href="#registering-a-deriver">Registering a Deriver</a></li></ul></li><li><a href="#attribute-guided-rewriting">Attribute-guided Rewriting</a><ul><li><a href="#the-list-of-attributes">The List of Attributes</a></li><li><a href="#the-expand-function_2">The Expand Function</a></li><li><a href="#creating-a-rewriting-rule">Creating a rewriting rule</a></li></ul></li><li><a href="#constant-rewriting">Constant Rewriting</a></li><li><a href="#special-functions">Special Functions</a></li></ul></li><li><a href="#global_transformation">Global transformation</a></li><li><a href="#inlining-transformations">Inlining Transformations</a></li><li><a href="#integration-with-dune">Integration with Dune</a></li><li><a href="#generatingcode">Defining AST Transformations</a></li></ul></nav></div><div class="odoc-content"><h2 id="defining-a-transformation"><a href="#defining-a-transformation" class="anchor"></a>Defining a Transformation</h2><p>For <code>ppxlib</code>, a transformation is a description of a way to modify a given AST into another one. A transformation can be:</p><ul><li>A context-free transformation, which only acts on a portion of the AST. In the <code>ppxlib</code> framework, those transformations are represented by values of type <a href="Ppxlib/Context_free/Rule/index.html#type-t" title="Ppxlib.Context_free.Rule.t"><code>Context_free.Rule.t</code></a> and are executed in the <a href="driver.html#context-free-phase" title="context-free-phase">context-free phase</a>. This is the strongly recommended kind of transformation due to its <a href="driver.html#advantages" title="advantages">important advantages</a>, such as good performance, well-defined composition semantics, and the safety and trustability that comes with well-isolated and strictly local modifications.</li><li>A global transformation, which takes the simple form of a function of type <code>structure -> structure</code> or <code>signature -> signature</code>, that can sometimes take extra information as additional arguments. Such a transformation is applied in the <a href="driver.html#global-transfo-phase" title="global-transfo-phase">global transformation phase</a>, unless it has a good reason to have been registered in another phase. While global transformations are a flexible and powerful tool in the OCaml ecosystem, they come with many <a href="#global_transformation" title="global_transformation">drawbacks</a> and should only be used when really necessary.</li></ul><p>In order to register a transformation to the <code>ppxlib</code> driver, one should use the <a href="Ppxlib/Driver/V2/index.html#val-register_transformation" title="Ppxlib.Driver.V2.register_transformation"><code>Driver.V2.register_transformation</code></a>. This function is used to register all rewriter types in every different phase, except derivers, which are abstracted away in <a href="Ppxlib/Deriving/index.html" title="Ppxlib.Deriving"><code>Deriving</code></a>.</p><h2 id="context-free-transformation"><a href="#context-free-transformation" class="anchor"></a>Context-Free Transformation</h2><p>In <code>ppxlib</code>, the type for context-free transformation is <a href="Ppxlib/Context_free/Rule/index.html#type-t" title="Ppxlib.Context_free.Rule.t"><code>Context_free.Rule.t</code></a>. Rules will be applied during the AST's top-down traverse of the context-free pass. A rule contains the information about when it should be applied in the traversal, as well as the transformation to apply.</p><p>Currently, rules can only be defined to apply in five different contexts:</p><ul><li>on extensions points, such as <code>[%ext_point payload]</code></li><li>on some structure or signature items with a deriving attribute, such as <code>type t = Nil [@@deriving show]</code>,</li><li>on AST nodes with attributes, such as <code>let x = 42 [@@attr]</code>,</li><li>on <a href="https://v2.ocaml.org/manual/extensionsyntax.html#ss:extension-literals">litterals with modifiers</a>, such as <code>41g</code> or <code>43.2x</code>,</li><li>on function application or identifiers, such as <code>meta_function "99"</code> and <code>meta_constant</code>.</li></ul><p>In order to define rules on extensions points, we will use the <a href="Ppxlib/Extension/index.html" title="Ppxlib.Extension"><code>Extension</code></a> module. In order to define deriving rules, we will use the <a href="Ppxlib/Deriving/index.html" title="Ppxlib.Deriving"><code>Deriving</code></a> module. For the three other rules, we will directly use the <a href="Ppxlib/Context_free/Rule/index.html" title="Ppxlib.Context_free.Rule"><code>Context_free.Rule</code></a> module.</p><h3 id="extenders"><a href="#extenders" class="anchor"></a>Extenders</h3><p>An <a href="driver.html#def_extenders" title="def_extenders">extender</a> is characterised by several things:</p><ul><li><p>The situation that triggers the rewriting, which consists of two things:</p><ul><li>The extension points' name on which it is triggered. For instance, an extender triggered on <code>[%name]</code> would not be triggered on <code>[%other_name]</code></li><li>The AST context on which it applies. Indeed, extension points can be used in many different places: expression, pattern, core type, etc., and the extender should be restricted to one context, as it produces code of a single type. So, an extender triggered on expressions could be triggered on <code>let x = [%name]</code> but not on <code>let [%name] = expr</code>.</li></ul></li><li><p>The actual rewriting of the extension node:</p><ul><li>A function, called "expander", taking arguments and outputting the generated AST</li><li>How to extract from the payload the arguments to pass to the expander</li></ul></li></ul><h4 id="ext_context"><a href="#ext_context" class="anchor"></a>The Extender Context</h4><p>The context is a value of type <a href="Ppxlib/Extension/Context/index.html#type-t" title="Ppxlib.Extension.Context.t"><code>Extension.Context.t</code></a>. For instance, to define an extender for expression-extension points, the correct context is <a href="Ppxlib/Extension/Context/index.html#val-expression" title="Ppxlib.Extension.Context.expression"><code>Extension.Context.expression</code></a>. Consult the <a href="Ppxlib/Extension/Context/index.html" title="Ppxlib.Extension.Context"><code>Extension.Context</code></a> module's API for the list of all contexts!</p><pre class="language-ocaml"><code> # let context = Extension.Context.expression;;
|
||
val context : expression Extension.Context.t =
|
||
Ppxlib.Extension.Context.Expression</code></pre><h4 id="the-extender-name"><a href="#the-extender-name" class="anchor"></a>The Extender Name</h4><p>The extension point name on which it applies is simply a string.</p><pre class="language-ocaml"><code> # let extender_name = "add_suffix" ;;
|
||
val extender_name : string = "add_suffix"</code></pre><p>See below for examples on when the above name and context will trigger rewriting:</p><pre class="language-ocaml"><code> (* will trigger rewriting: *)
|
||
let _ = [%add_suffix "payload"]
|
||
|
||
(* won't trigger rewriting: *)
|
||
let _ = [%other_name "payload"] (* wrong name *)
|
||
let _ = match () with [%add_suffix "payload"] -> () (* wrong context *)</code></pre><h4 id="the-payload-extraction"><a href="#the-payload-extraction" class="anchor"></a>The Payload Extraction</h4><p>An extension node contains a <a href="Astlib/Ast_502/Parsetree/index.html#type-payload" title="Ppxlib.Parsetree.payload"><code>payload</code></a>, which will be passed to the transformation function. However, while this payload contains all information, it is not always structured the best way for the transformation function. For instance, in <code>[%add_suffix "payload"]</code>, the string <code>"payload"</code> is encoded as a structure item consisting of an expression’s evaluation, a constant that is a string.</p><p><code>ppxlib</code> allows separating the transformation function from the extraction of the payload’s relevant information. As explained in depth in the <a href="matching-code.html" title="matching-code">Destructing AST nodes</a> chapter, this extraction is done by destructing the payload’s structure (which is therefore restricted: <code>[%add_suffix 12]</code> would be refused by the rewriter of the example below). The extraction is defined by a value of type <a href="Ppxlib/Ast_pattern/index.html#type-t" title="Ppxlib.Ast_pattern.t"><code>Ast_pattern.t</code></a>. The <a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code></a> module provides some kind of pattern-matching on AST nodes: a way to structurally extract values from an AST node in order to generate a value of another kind.</p><p>For instance, a value of type <code>(payload, int -> float -> expression, expression) Ast_pattern.t</code> means that it defines a way to extract an <code>int</code> and a <code>float</code> from a <a href="Astlib/Ast_502/Parsetree/index.html#type-payload" title="Ppxlib.Parsetree.payload"><code>payload</code></a>, which should be then combined to define a value of type <a href="Astlib/Ast_502/Parsetree/index.html#type-expression" title="Ppxlib.Parsetree.expression"><code>expression</code></a>.</p><p>In our case, the matched value will always be a <a href="Astlib/Ast_502/Parsetree/index.html#type-payload" title="Ppxlib.Parsetree.payload"><code>payload</code></a>, as that's the type for extension points' payloads. The type of the produced node will have to match the <a href="#ext_context" title="ext_context">type of extension node we rewrite</a>, <a href="Astlib/Ast_502/Parsetree/index.html#type-expression" title="Ppxlib.Parsetree.expression"><code>expression</code></a> in our example.</p><pre class="language-ocaml"><code> # let extracter () = Ast_pattern.(single_expr_payload (estring __)) ;;
|
||
val extracter : unit -> (payload, string -> 'a, 'a) Ast_pattern.t = <fun></code></pre><p>The above pattern extracts a string inside an extension node pattern. It will extract <code>"string"</code> in the the extension node <code>[%ext_name "string"]</code> and will refuse <code>[%ext_name 1+1]</code>. For other ready-to-use examples of patterns, refer to the <a href="matching-code.html#pattern_examples" title="pattern_examples">example</a> section. For more in-depth explanation on the types and functions used above, see the <a href="matching-code.html" title="matching-code">Destructing AST nodes</a> chapter and the <a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code> API</a> .</p><p>The unit argument in <code>extractor</code> is not important. It is added so that <a href="https://v2.ocaml.org/manual/polymorphism.html#ss:valuerestriction">value restriction</a> does not add noise to the type variables.</p><h4 id="the-expand-function"><a href="#the-expand-function" class="anchor"></a>The Expand Function</h4><p>The expander is the function that takes the values extracted from the payload and produces the value that replaces the extension node.</p><p>Building and inspecting AST nodes can be painful due to how <a href="Astlib/Ast_502/Parsetree/index.html" title="Ppxlib.Parsetree">large</a> the AST type is. <code>ppxlib</code> provides several helper modules to ease this generation, such as <a href="Ppxlib/Ast_builder/index.html" title="Ppxlib.Ast_builder"><code>Ast_builder</code></a>, <a href="Ppxlib_metaquot/index.html"><code>Ppxlib_metaquot</code></a>, <a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code></a>, and <a href="Ppxlib/Ast_traverse/index.html" title="Ppxlib.Ast_traverse"><code>Ast_traverse</code></a>, which are explained in their own chapters: <a href="generating-code.html" title="generating-code">Generating AST nodes</a>, <a href="matching-code.html" title="matching-code">Destructing AST nodes</a> and <a href="ast-traversal.html" title="ast-traversal">Traversing AST nodes</a>.</p><p>In the example below, you can ignore the body of the function until reading those chapters.</p><pre class="language-ocaml"><code> # let expander ~ctxt s =
|
||
let loc = Expansion_context.Extension.extension_point_loc ctxt in
|
||
Ast_builder.Default.(estring ~loc (s ^ "_suffixed")) ;;
|
||
val expander : ctxt:Expansion_context.Extension.t -> string -> expression =
|
||
<fun></code></pre><p>The expander takes <code>ctxt</code> as a named argument that is ignored here. This argument corresponds to additional information, such as the location of the extension node. More precisely, it is of type <a href="Ppxlib/Expansion_context/Extension/index.html#type-t" title="Ppxlib.Expansion_context.Extension.t"><code>Expansion_context.Extension.t</code></a> and includes:</p><ul><li>The location of the extension node</li><li>The tool that called the rewriting (<code>merlin</code>, <code>ocamlc</code>, <code>ocaml</code>, <code>ocamlopt</code>, etc.)</li><li>The name of the input file given to the driver (see <a href="Ppxlib/Expansion_context/Base/index.html#val-input_name" title="Ppxlib.Expansion_context.Base.input_name"><code>Expansion_context.Base.input_name</code></a>)</li><li>The <code>code_path</code> (see <a href="Ppxlib/Expansion_context/Base/index.html#val-input_name" title="Ppxlib.Expansion_context.Base.input_name"><code>Expansion_context.Base.input_name</code></a> and <a href="Ppxlib/Code_path/index.html" title="Ppxlib.Code_path"><code>Code_path</code></a>)</li></ul><h4 id="declaring-an-extender"><a href="#declaring-an-extender" class="anchor"></a>Declaring an Extender</h4><p>When we have defined the four prerequisites, we are able to combine all of them to define an extender using the <a href="Ppxlib/Extension/V3/index.html#val-declare" title="Ppxlib.Extension.V3.declare"><code>Extension.V3.declare</code></a> function.</p><pre class="language-ocaml"><code> # V3.declare ;;
|
||
string ->
|
||
'context Context.t ->
|
||
(payload, 'a, 'context) Ast_pattern.t ->
|
||
(ctxt:Expansion_context.Extension.t -> 'a) ->
|
||
t</code></pre><p>Note that the type is consistent: the context on which the expander is applied and the value produced by the expander need to be equal (indeed, <code>'a</code> must be of the form <code>'extacted_1 -> 'extracted_2 -> ... -> 'context</code> with the constraints given by <a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code></a>).</p><p>We are thus able to create the extender given by the previous examples:</p><pre class="language-ocaml"><code> # let my_extender = Extension.V3.declare extender_name context (extracter()) expander ;;
|
||
val my_extender : Extension.t = <abstr></code></pre><p>Note that we use the <code>V3</code> version of the <code>declare</code> function, which passes the expansion context to the expander. Previous versions were kept for retro-compatibility.</p><p>We can finally turn the extender into a rule (using <a href="Ppxlib/Context_free/Rule/index.html#val-extension" title="Ppxlib.Context_free.Rule.extension"><code>Context_free.Rule.extension</code></a>) and register it to the driver:</p><pre class="language-ocaml"><code> # let extender_rule = Context_free.Rule.extension my_extender ;;
|
||
val extender_rule : Context_free.Rule.t = <abstr>
|
||
# Driver.register_transformation ~rules:[extender_rule] "name_only_for_debug_purpose" ;;
|
||
- : unit = ()</code></pre><p>Now, the following:</p><pre class="language-ocaml"><code> let () = print_endline [%add_suffix "helloworld"]</code></pre><p>would be rewritten by the PPX in:</p><pre class="language-ocaml"><code> let () = print_endline "helloworld_suffixed"</code></pre><h3 id="derivers"><a href="#derivers" class="anchor"></a>Derivers</h3><p>A <a href="driver.html#def_derivers" title="def_derivers">deriver</a> is characterised by several things:</p><ul><li>The way to parse arguments passed through the attribute payload</li><li>The set of other derivers that need to run before it is applied</li><li>The actual generator function</li></ul><p>Contrary to extenders, the registration of the deriver as a <a href="Ppxlib/Context_free/Rule/index.html#type-t" title="Ppxlib.Context_free.Rule.t"><code>Context_free.Rule.t</code></a> is not made by the user via <a href="Ppxlib/Driver/index.html#val-register_transformation" title="Ppxlib.Driver.register_transformation"><code>Driver.register_transformation</code></a>, but rather by <a href="Ppxlib/Deriving/index.html#val-add" title="Ppxlib.Deriving.add"><code>Deriving.add</code></a>.</p><h5 id="derivers-arguments"><a href="#derivers-arguments" class="anchor"></a>Derivers Arguments</h5><p>In <code>ppxlib</code>, a deriver is applied by adding an attribute containing the derivers' names to apply:</p><pre class="language-ocaml"><code> type tree = Leaf | Node of tree * tree [@@deriving show, yojson]</code></pre><p>However, it is also possible to pass arguments to the derivers, either through a record or through labelled arguments:</p><pre class="language-ocaml"><code> type tree = Leaf | Node of tree * tree [@@deriving my_deriver ~flag ~option1:52]</code></pre><p>or</p><pre class="language-ocaml"><code> type tree = Leaf | Node of tree * tree [@@deriving my_deriver { flag; option1=52 }]</code></pre><p>The <code>flag</code> argument is a flag, and it can only be present or absent but not take a value. The <code>option1</code> argument is a regular argument, so it is also optional but can take a value.</p><p>In <code>ppxlib</code>, arguments have the type <a href="Ppxlib/Deriving/Args/index.html#type-t" title="Ppxlib.Deriving.Args.t"><code>Deriving.Args.t</code></a>. Similarly to the <a href="Ppxlib/Ast_pattern/index.html#type-t" title="Ppxlib.Ast_pattern.t"><code>Ast_pattern.t</code></a> type, a value of type <code>(int -> string -> structure, structure) Args.t</code> means that it provides a way to extract an integer from the argument and a string from the options, later combined to create a structure.</p><p>The way to define a <a href="Ppxlib/Deriving/Args/index.html#type-t" title="Ppxlib.Deriving.Args.t"><code>Deriving.Args.t</code></a> value is to start with the value describing an empty set of arguments, <a href="Ppxlib/Deriving/Args/index.html#val-empty" title="Ppxlib.Deriving.Args.empty"><code>Deriving.Args.empty</code></a>. Then add the arguments one by one, using the combinator <a href="Ppxlib/Deriving/Args/index.html#val-(+>)" title="Ppxlib.Deriving.Args.(+>)"><code>Deriving.Args.(+>)</code></a>. Each argument is created using either <a href="Ppxlib/Deriving/Args/index.html#val-arg" title="Ppxlib.Deriving.Args.arg"><code>Deriving.Args.arg</code></a> for optional arguments (with value extracted using <a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code></a>) or <a href="Ppxlib/Deriving/Args/index.html#val-flag" title="Ppxlib.Deriving.Args.flag"><code>Deriving.Args.flag</code></a> for optional arguments without values.</p><pre class="language-ocaml"><code> # let args () = Deriving.Args.(empty +> arg "option1" (eint __) +> flag "flag") ;;
|
||
val args : (int option -> bool -> 'a, 'a) Deriving.Args.t = <abstr></code></pre><h5 id="derivers-dependency"><a href="#derivers-dependency" class="anchor"></a>Derivers Dependency</h5><p><code>ppxlib</code> allows declaring that a deriver depends on the previous application of another deriver. This is expressed simply as a list of derivers. For instance, the <a href="https://github.com/janestreet/ppx_csv_conv">csv</a> deriver depends on the <a href="https://github.com/janestreet/ppx_fields_conv">fields</a> deriver to run first.</p><pre class="language-ocaml"><code> # let deps = [] ;;
|
||
val deps : 'a list = []</code></pre><p>In this example, we do not include any dependency.</p><h4 id="generator-function"><a href="#generator-function" class="anchor"></a>Generator Function</h4><p>Similarly to an extender's <code>expand</code> function, the function generating new code in derivers also takes a context and the arguments extracted from the attribute payload. Here again, the body of the example function can be safely ignored ,as it relies on <a href="generating-code.html" title="generating-code">later chapters</a>.</p><pre class="language-ocaml"><code> # let generate_impl ~ctxt _ast option1 flag =
|
||
let return s = (* See "Generating code" chapter *)
|
||
let loc = Expansion_context.Deriver.derived_item_loc ctxt in
|
||
[ Ast_builder.Default.(pstr_eval ~loc (estring ~loc s) []) ]
|
||
in
|
||
if flag then return "flag is on"
|
||
else
|
||
match option1 with
|
||
| Some i -> return (Printf.sprintf "option is %d" i)
|
||
| None -> return "flag and option are not set" ;;
|
||
val generate_impl :
|
||
ctxt:Expansion_context.Deriver.t ->
|
||
'a -> int option -> bool -> structure_item list = <fun></code></pre><p>Similarly to extenders, there is an additional (ignored in the example) argument to the function: the context. This time, the context is of type <a href="Ppxlib/Expansion_context/Deriver/index.html#type-t" title="Ppxlib.Expansion_context.Deriver.t"><code>Expansion_context.Deriver.t</code></a> and includes:</p><ul><li>The location of the derived item</li><li>Whether the code generation will be inlined (see <a href="#inlining-transformations" title="inlining-transformations">Inlining Transformations</a>)</li><li>The tool that called the rewriting (<code>merlin</code>, <code>ocamlc</code>, <code>ocaml</code>, <code>ocamlopt</code>, etc.),</li><li>The name of the input file given to the driver (see <a href="Ppxlib/Expansion_context/Base/index.html#val-input_name" title="Ppxlib.Expansion_context.Base.input_name"><code>Expansion_context.Base.input_name</code></a>)</li><li>The <code>code_path</code> (see <a href="Ppxlib/Expansion_context/Base/index.html#val-input_name" title="Ppxlib.Expansion_context.Base.input_name"><code>Expansion_context.Base.input_name</code></a> and <a href="Ppxlib/Code_path/index.html" title="Ppxlib.Code_path"><code>Code_path</code></a>).</li></ul><h4 id="registering-a-deriver"><a href="#registering-a-deriver" class="anchor"></a>Registering a Deriver</h4><p>Once the generator function is defined, we can combine the argument extraction and the generator function to create a <a href="Ppxlib/Deriving/Generator/index.html#type-t" title="Ppxlib.Deriving.Generator.t"><code>Deriving.Generator.t</code></a>:</p><pre class="language-ocaml"><code> # let generator () = Deriving.Generator.V2.make (args()) generate_impl ;;
|
||
val generator : unit -> (structure_item list, 'a) Deriving.Generator.t = <abstr></code></pre><p>This generator can then be registered as a deriver through the <a href="Ppxlib/Deriving/index.html#val-add" title="Ppxlib.Deriving.add"><code>Deriving.add</code></a> function. Note that, <a href="Ppxlib/Deriving/index.html#val-add" title="Ppxlib.Deriving.add"><code>Deriving.add</code></a> will call <a href="Ppxlib/Driver/index.html#val-register_transformation" title="Ppxlib.Driver.register_transformation"><code>Driver.register_transformation</code></a> itself, so you won't need to do it manually. Adding a deriver is done in a way that no two derivers with the same name can be registered. This includes derivers registered through the <a href="https://github.com/ocaml-ppx/ppx_deriving">ppx_deriving</a> library.</p><pre class="language-ocaml"><code> # let my_deriver = Deriving.add "my_deriver" ~str_type_decl:(generator()) ;;
|
||
val my_deriver : Deriving.t = <abstr></code></pre><p>The different, optional named argument allows registering generators to be applied in different contexts and in one function call. Remember that you can only add one deriver with a given name, even if applied on different contexts. As the API shows, derivers are restricted to being applied in the following contexts:</p><ul><li>Type declarations (<code>type t = Foo of int</code>)</li><li>Type extensions (<code>type t += Foo of int</code>)</li><li>Exceptions (<code>exception E of int</code>)</li><li>Module type declarations (<code>module type T = sig end</code>)</li></ul><p>in both structures and signatures.</p><h3 id="attribute-guided-rewriting"><a href="#attribute-guided-rewriting" class="anchor"></a>Attribute-guided Rewriting</h3><p><code>ppxlib</code> provides context-free rules that, like derivers, apply to nodes based on their attributes but, like extenders, allow rewriting the entire AST node. These provide lighter-weight syntax than extenders but that also means it's less obvious that they're rewriting the syntax tree.</p><p>Before using this kind of rule, carefully consider using an extender instead. <code>ppxlib</code> provides an opinionated syntax for preprocessors so that it's easy for users to understand what code is being affected by the PPX. In general, these should only be used to slightly modify the node the attribute is attached to, rather than rewrite it to something new. The syntax of extenders highlights to users where more involved rewriting is taking place.</p><p>These are composed of:</p><ul><li>The name of the rewrite rule</li><li>The list of attributes they define</li><li>The expand function</li></ul><p>They are defined to apply in a specific context, specifically, they can be registered to be processed in the same contexts as extenders can occur.</p><h4 id="the-list-of-attributes"><a href="#the-list-of-attributes" class="anchor"></a>The List of Attributes</h4><p>A given rewrite rule can have multiple attributes that trigger it, if any of the attributes are present on a single node then the rule is triggered and provided with the AST node along with the payload of all the attributes registered by this rule. To declare attributes use the <a href="Ppxlib/Attribute/index.html#val-declare" title="Ppxlib.Attribute.declare"><code>Attribute.declare</code></a> function (or the other similar functions in that module). Note that the <a href="Ppxlib/Attribute/Context/index.html" title="Ppxlib.Attribute.Context"><code>Context.t</code></a> must match the type of AST nodes that the rule will apply to.</p><pre class="language-ocaml"><code> # let prefix_attr = Attribute.declare "example.prefix" Expression
|
||
Ast_pattern.(single_expr_payload (estring __)) Fun.id
|
||
and suffix_attr = Attribute.declare "example.suffix" Expression
|
||
Ast_pattern.(single_expr_payload (estring __)) Fun.id ;;
|
||
val prefix_attr : (expression, string) Attribute.t = <abstr>
|
||
val suffix_attr : (expression, string) Attribute.t = <abstr></code></pre><h4 id="the-expand-function_2"><a href="#the-expand-function_2" class="anchor"></a>The Expand Function</h4><p>The expand function takes the AST node (with this rule's attributes already stripped) and the payloads of all the declared attributes (as a list of <code>payload option</code> to allow for attributes that haven't been included).</p><pre class="language-ocaml"><code> # let expander
|
||
~ctxt
|
||
expression
|
||
([ prefix; suffix ] : _ Context_free.Rule.Parsed_payload_list.t)
|
||
=
|
||
match expression.pexp_desc with
|
||
| Pexp_ident { txt = Lident name; loc } ->
|
||
let prefixed = Option.value ~default:"" prefix ^ name in
|
||
let suffixed = prefixed ^ Option.value ~default:"" suffix in
|
||
{ expression with pexp_desc = Pexp_ident { txt = Lident suffixed; loc } }
|
||
| _ -> expression ;;
|
||
val expander :
|
||
ctxt:'a ->
|
||
expression ->
|
||
(string * (string * unit)) Context_free.Rule.Parsed_payload_list.t -> expression =
|
||
<fun></code></pre><h4 id="creating-a-rewriting-rule"><a href="#creating-a-rewriting-rule" class="anchor"></a>Creating a rewriting rule</h4><p>Finally, we can create the rule using the appropriate <a href="Ppxlib/Extension/Context/index.html" title="Ppxlib.Extension.Context"><code>Ppxlib.Extension.Context</code></a> and register it with the driver using <span class="xref-unresolved" title="Ppxlib.Context_free.Rule.attr_multiple_replace"><code>Context_free.Rule.attr_multiple_replace</code></span>. There's also a <a href="Ppxlib/Context_free/Rule/index.html#val-attr_replace" title="Ppxlib.Context_free.Rule.attr_replace"><code>Context_free.Rule.attr_replace</code></a> function with a slightly simpler API if you only use a single attribute.</p><pre class="language-ocaml"><code> # let rewrite_rule = Context_free.Rule.attr_multiple_replace "example" Expression
|
||
[ prefix_attr; suffix_attr ] expander ;;
|
||
val rule : Context_free.Rule.t = <abstr>
|
||
# Driver.register_transformation ~rules:[rewrite_rule] "example" ;;
|
||
- : unit = ()</code></pre><p>Now, for example, the following:</p><pre class="language-ocaml"><code> let _ = foo [@prefix "p_"] [@suffix "_s"]</code></pre><p>will be rewritten to:</p><pre class="language-ocaml"><code> let _ = p_foo_s</code></pre><h3 id="constant-rewriting"><a href="#constant-rewriting" class="anchor"></a>Constant Rewriting</h3><p>OCaml integrates a <a href="https://v2.ocaml.org/manual/extensionsyntax.html#ss:extension-literals">syntax</a> to define special constants. Any <code>g..z</code> or <code>G..Z</code> suffix appended after a float or int is accepted by the parser (but refused later by the compiler). This means a PPX must rewrite them.</p><p><code>ppxlib</code> provides the <a href="Ppxlib/Context_free/Rule/index.html#val-constant" title="Ppxlib.Context_free.Rule.constant"><code>Context_free.Rule.constant</code></a> function to rewrite those litteral constants. The character (between <code>g</code> and <code>z</code> or <code>G</code> and <code>Z</code>) has to be provided, as well as the constant kind (float or int), and both the location and the litteral as a string will be passed to a rewriting function:</p><pre class="language-ocaml"><code> # let kind = Context_free.Rule.Constant_kind.Integer ;;
|
||
val kind : Context_free.Rule.Constant_kind.t =
|
||
Ppxlib.Context_free.Rule.Constant_kind.Integer
|
||
# let rewriter loc s = Ast_builder.Default.eint ~loc (int_of_string s * 100) ;;
|
||
val rewriter : location -> string -> expression = <fun>
|
||
# let rule = Context_free.Rule.constant kind 'g' rewriter ;;
|
||
val rule : Context_free.Rule.t = <abstr>
|
||
# Driver.register_transformation ~rules:[ rule ] "constant" ;;
|
||
- : unit = ()</code></pre><p>As an example with the above transformation, <code>let x = 2g + 3g</code> will be rewritten to <code>let x = 200 + 300</code>.</p><h3 id="special-functions"><a href="#special-functions" class="anchor"></a>Special Functions</h3><p><code>ppxlib</code> supports registering functions to be applied at compile time. A registered identifier <code>f_macro</code> will trigger rewriting in two situations:</p><ol><li>When it plays the role of the function in a function application</li><li>Anywhere it appears in an expression</li></ol><p>For instance, in</p><pre class="language-ocaml"><code> let _ = (f_macro arg1 arg2, f_macro)</code></pre><p>the rewriting will be triggered once for the left-hand side <code>f_macro arg1 arg2</code> and once for the right hand side <code>f_macro</code>. It is the expansion function that is responsible for distinguishing between the two cases: using pattern-matching to distinguish between a function application in one case and a single identifier in the other.</p><p>In order to register a special function, one needs to use <a href="Ppxlib/Context_free/Rule/index.html#val-special_function" title="Ppxlib.Context_free.Rule.special_function"><code>Context_free.Rule.special_function</code></a>, indicating the name of the special function and the rewriter. The rewriter will take the expression (without expansion context) and should output an <code>expression option</code>, where:</p><ul><li><code>None</code> signifies that no rewriting should be done: the top-down pass can continue (potentially inside the expression).</li><li><code>Some exp</code> signifies the original expression should be replaced by <code>expr</code>. The top-down pass continues with <code>expr</code>.</li></ul><p>The difference between <code>fun expr -> None</code> and <code>fun expr -> Some expr</code> is that the former will continue the top-down pass <em>inside</em> <code>expr</code>, while the latter will continue the top-down pass from <code>expr</code> (included), therefore starting an infinite loop.</p><pre class="language-ocaml"><code> # let expand e =
|
||
let return n = Some (Ast_builder.Default.eint ~loc:e.pexp_loc n) in
|
||
match e.pexp_desc with
|
||
| Pexp_apply (_, arg_list) -> return (List.length arg_list)
|
||
| _ -> return 0
|
||
;;
|
||
val expand : expression -> expression option = <fun>
|
||
# let rule = Context_free.Rule.special_function "n_args" expand ;;
|
||
val rule : Context_free.Rule.t = <abstr>
|
||
# Driver.register_transformation ~rules:[ rule ] "special_function_demo" ;;
|
||
- : unit = ()</code></pre><p>With such a rewriter registered:</p><pre class="language-ocaml"><code> # Printf.printf "n_args is applied with %d arguments\n" (n_args ignored "arguments");;
|
||
n_args is applied with 2 arguments
|
||
- : unit = ()</code></pre><h2 id="global_transformation"><a href="#global_transformation" class="anchor"></a>Global transformation</h2><p>Global transformations are the most general kind of transformation. As such, they allow doing virtually any modifications, but this comes with several drawbacks. There are very few PPXs that really need this powerful but dangerous feature. In fact, even if, at first sight, it seems like your transformation isn't context-free, it's likely that you can find a more suitable abstraction with which it becomes context-free. Whenever that's the case, go for context-free! The mentioned drawbacks are:</p><ul><li>It is harder for the user to know exactly what parts of the AST will be changed. Your transformation becomes a scary black box.</li><li>It is harder for <code>ppxlib</code> to combine several global transformations, as there is no guarantee that the effect of one will work well with the effect of another.</li><li>The job done by two global transformations (e.g., an AST traverse) cannot be factorised, resulting in slower compilation time.</li><li>If you don't make sure that you really follow all <a href="good-practices.html" title="good-practices">good practices</a>, you might end up messing up the global developer experience.</li></ul><p>For all these reasons, a global transformation should be avoided whenever a context-free transformation could do the job, which by experience seems to be most of the time. The API for defining a global transformation is easy. A global transformation consists simply of the function and can be directly be registered with <a href="Ppxlib/Driver/index.html#val-register_transformation" title="Ppxlib.Driver.register_transformation"><code>Driver.register_transformation</code></a>.</p><pre class="language-ocaml"><code> # let f str = List.filter (fun _ -> Random.bool ()) str;; (* Randomly omit structure items *)
|
||
val f : 'a list -> 'a list = <fun>
|
||
# Driver.register_transformation ~impl:f "absent_minded_transformation"
|
||
- : unit = ()</code></pre><h2 id="inlining-transformations"><a href="#inlining-transformations" class="anchor"></a>Inlining Transformations</h2><p>When using a PPX, the transformation happens at compile time, and the produced code could be directly inlined into the original code. This allows dropping the dependency on <code>ppxlib</code> and the PPX used to generate the code.</p><p>This mechanism is implemented for derivers implemented in <code>ppxlib</code> and is convenient to use, especially in conjunction with Dune. When applying a deriver, using <code>[@@deriving_inline deriver_name]</code> will apply the inline mode of <code>deriver_name</code> instead of the normal mode.</p><p>Inline derivers will generate a <code>.corrected</code> version of the file that Dune can use to promote your file. For more information on how to use this feature to remove a dependency on <code>ppxlib</code> and a specific PPX from your project, refer to <a href="https://ocaml.org/docs/metaprogramming#dropping-ppxs-dependency-with-derivinginline">this guide</a>.</p><p>In addition to <code>[@@deriving_inline]</code>, there is also <code>[@@@expand_inline <structure payload>]</code> and <code>[@@@expand_inline: <signature payload>]</code>. These can be use to inline code generated by other context free transformations (not just derivers):</p><pre class="language-ocaml"><code> [@@@expand_inline let _ = [%add_suffix "foo"]]
|
||
|
||
let _ = "foo_suffixed"
|
||
|
||
[@@@end]
|
||
|
||
module type S = sig
|
||
[@@@expand_inline: type foo = [%pair_of string]]
|
||
|
||
type foo = string * string
|
||
|
||
[@@@end]
|
||
end</code></pre><h2 id="integration-with-dune"><a href="#integration-with-dune" class="anchor"></a>Integration with Dune</h2><p>If your PPX is written as a Dune project, you'll need to specify the <code>kind</code> field in your <code>dune</code> file with one of the following two values:</p><ul><li><code>ppx_rewriter</code>, or</li><li><code>ppx_deriver</code>.</li></ul><p>If your transformation is anything but a deriver (e.g. an extension node rewriter), use <code>ppx_rewriter</code>. If your transformation is a deriver, then the TLDR workflow is: use <code>ppx_deriver</code> and furthermore add <code>ppx_deriving</code> to your dependencies, i.e. to the <code>libraries</code> field of your dune file. In fact, the situation is quite a bit more complex, though: apart from applying the registered transformations, the Ppxlib driver also does several checks. One of those consists in checking the following: whenever the source code contains <code>[@@deriving foo (...)]</code>, then the Ppxlib driver expects a driver named <code>foo</code> to be registered. That's helpful to catch typos and missing dependencies on derivers and is certainly more hygienic than silently ignoring the annotation. However, for that check to work, the registered derivers must be grouped together into one process, i.e. a driver. UTop cannot use a static driver such as the Ppxlib one because dependencies are added dynamically to a UTop session. So the solution is the following: if you use <code>ppx_deriver</code> in your <code>kind</code> field, dune will add the right data to your PPXs META file to ensure that UTop will use the <code>ppx_deriving</code> driver, which links the derivers dynamically. As a result, <code>ppx_derivng</code> appears as a dependency in the META file. Therefore, whenever a user uses <code>ocamlfind</code> (e.g. by using UTop), they will hit an "<code>ppx_derivng</code> not found" error, unless you define <code>ppx_deriving</code> in your dependencies. So, long story short: if you strongly care about avoiding <code>ppx_deriving</code> as a dependency, use <code>ppx_rewriter</code> in your <code>kind</code> field and be aware of the fact that users won't be able to try your deriver in UTop; otherwise do the TLDR workflow.</p><p>Here is a minimal Dune stanza for a rewriter:</p><pre class="language-dune"><code> (library
|
||
(public_name my_ppx_rewriter)
|
||
(kind ppx_rewriter)
|
||
(libraries ppxlib))</code></pre><p>The public name you chose is the name your users will refer to your PPX in the <code>preprocess</code> field. For example, to use this PPX rewriter, one would add the <code>(preprocess (pps my_ppx_rewriter))</code> to their <code>library</code> or <code>executable</code> stanza.</p><h2 id="generatingcode"><a href="#generatingcode" class="anchor"></a>Defining AST Transformations</h2><p>In this chapter, we only focused on the <code>ppxlib</code> ceremony to declare all kinds of transformations. However, we did not cover how to write the actual generative function, the backbone of the transformation. <code>ppxlib</code> provides several modules to help with code generation and matching, which are covered in more depth in the next chapters of this documentation:</p><ul><li><a href="Ppxlib/Ast_traverse/index.html" title="Ppxlib.Ast_traverse"><code>Ast_traverse</code></a>, which helps in defining AST traversals, such as maps, folds, iter, etc.</li><li><a href="Ppxlib_ast/Ast_helper/index.html" title="Ppxlib.Ast_helper"><code>Ast_helper</code></a> and <a href="Ppxlib/Ast_builder/index.html" title="Ppxlib.Ast_builder"><code>Ast_builder</code></a>, for generating AST nodes in a simpler way than directly dealing with the <a href="Astlib/Ast_502/Parsetree/index.html" title="Ppxlib.Parsetree"><code>Parsetree</code></a> types, providing a more stable API.</li><li><a href="Ppxlib/Ast_pattern/index.html" title="Ppxlib.Ast_pattern"><code>Ast_pattern</code></a>, the sibling of <a href="Ppxlib/Ast_builder/index.html" title="Ppxlib.Ast_builder"><code>Ast_builder</code></a> for matching on AST nodes, extracting values for them.</li><li><a href="Ppxlib_metaquot/index.html"><code>Ppxlib_metaquot</code></a>, a PPX to manipulate code more simply by quoting and unquoting code.</li></ul><p>This documentation also includes some <a href="good-practices.html" title="good-practices">guidelines</a> on how to generate nice code. We encourage you to read and follow it to produce high quality PPXs:</p><ul><li>A section on good <a href="good-practices.html#handling_errors" title="handling_errors">error reporting</a></li><li>A section on the <a href="good-practices.html#quoting" title="quoting">mechanism</a></li><li>A section on how to <a href="good-practices.html#testing-your-ppx" title="testing-your-ppx">test</a> your PPX</li><li>A section on how to collaborate with Merlin effectively by being careful with <a href="good-practices.html#testing-your-ppx" title="testing-your-ppx">locations</a></li></ul><p> <div style="display: flex; justify-content:space-between"><div><a href="driver.html" title="driver">< The Driver</a> </div><div><a href="generating-code.html" title="generating-code">Generating AST nodes ></a> </div></div></p></div></body></html>
|