Posts tagged "consult":

30 Mar 2023

More on tree-sitter and consult

Here's a few things that I've gotten help with figuring out during the last few days. Both things are related to my playing with tree-sitter that I've written about earlier, here and here.

You might also be interested in the two repositories where the full code is. (I've linked to the specific commits as of this writing.)

Anonymous nodes and matching in tree-sitter

In the grammar for Cabal I have a rule for sections that like this

sections: $ => repeat1(choice(
    $.benchmark,
    $.common,
    $.executable,
    $.flag,
    $.library,
    $.source_repository,
    $.test_suite,
)),

where each section followed this pattern

benchmark: $ => seq(
    repeat($.comment),
    'benchmark',
    field('name', $.section_name),
    field('properties', $.property_block),
),

This made it a little bit difficult to capture the relevant parts of each section to implement consult-cabal. I thought a pattern like this ought to work

(cabal
 (sections
  (_ _ @type
     name: (section_name)? @name)))

but it didn't; I got way too many things captured in type. Clearly I had misunderstood something about the wildcards, or the query syntax. I attempted to add a field name to the anonymous node, i.e. change the sections rules like this

benchmark: $ => seq(
    repeat($.comment),
    field('type', 'benchmark'),
    field('name', $.section_name),
    field('properties', $.property_block),
),

It was accepted by tree-sitter generate, but the field type was nowhere to be found in the parse tree.

Then I changed the query to list the anonymous nodes explicitly, like this

(cabal
 (sections
  (_ ["benchmark" "common" "executable" ...] @type
     name: (section_name)? @name)))

That worked, but listing all the sections like that in the query didn't sit right with me.

Luckily there's a discussions area in tree-sitters GitHub so a fairly short discussion later I had answers to why my query behaved like it did and a solution that would allow me to not list all the section types in the query. The trick is to wrap the string in a call to alias to make it a named node. After that it works to add a field name to it as well, of course. The section rules now look like this

benchmark: $ => seq(
    repeat($.comment),
    field('type', alias('benchmark', $.section_type)),
    field('name', $.section_name),
    field('properties', $.property_block),
),

and the final query looks like this

(cabal
 (sections
  (_
   type: (section_type) @type
   name: (section_name)? @name)))

With that in place I could improve on the function that collects all the items for consult-cabal so it now show the section's type and name instead of the string representation of the tree-sitter node.

State in a consult source for preview of lines in a buffer

I was struggling with figuring out how to make a good state function in order to preview the items in consult-cabal. The GitHub repo for consult doesn't have discussions enabled, but after a discussion in an issue I'd arrived at a state function that works very well.

The state function makes use of functions in consult and looks like this

(defun consult-cabal--state ()
  "Create a state function for previewing sections."
  (let ((state (consult--jump-state)))
    (lambda (action cand)
      (when cand
        (let ((pos (get-text-property 0 'section-pos cand)))
          (funcall state action pos))))))

The trick here was to figure out how the function returned by consult--jump-state actually works. On the surface it looks like it takes an action and a candidate, (lambda (action cand) ...). However, the argument cand shouldn't be the currently selected item, but rather a postion (ideally a marker), so I had to attach another text property on the items (section-pos, which is fetched in the inner lambda). This position is then what's passed to the function returned by consult--jump-state.

In hindsight it seems so easy, but I was struggling with this for an entire evening before finally asking the question the morning after.

Tags: consult emacs tree-sitter
27 Mar 2023

Cabal, tree-sitter, and consult

After my last post I thought I'd move on to implement the rest of the functions in haskell-mode's major mode for Cabal, functions like haskell-cabal-goto-library-section and haskell-cabal-goto-executable-section. Then I realised that what I really want is a way to quickly jump to any section, that is, I want consult-cabal!

What follows is very much a work-in-progress, but hopefully it'll show enough promise.

Listing the sections

As I have a tree-sitter parse tree to hand it is fairly easy to fetch all the nodes corresponding to sections. Since the last post I've made some improvements to the parser and now the parse tree looks like this (I can recommend the function treesit-explore-mode to expore the parse tree, I've found it invaluable ever since I realised it existed)

(cabal
 ...
 (properties ...)
 (sections
  (common common (section_name) ...)
  (library library ...)
  (executable executable (section_name) ...)
  ...))

That is, all the sections are children of the node called sections.

The function to use for fetching all the nodes is treesit-query-capture, it needs a node to start on, which this case should be the full parse tree, i.e. (treesit-buffer-root-node 'cabal) and a query string. Given the structure of the parse tree, and that I want to capture all children of sections, a query string like this one works

"(cabal (sections (_)* @section))"

Finally, by default treesit-query-capture returns a list of tuples of the form (<capture> . <node>), but in this case I only want the list of nodes, so the full call will look like this

(treesit-query-capture (treesit-buffer-root-node 'cabal)
                       "(cabal (sections (_)* @section))"
                       nil nil t)

Hooking it up to consult

As I envision adding more things to jump to in the future, I decided to make use of consult--multi. That in turn means I need to define a "source" for the sections. After a bit of digging and rummaging in the consult source I put together this

(defvar consult-cabal--source-section
  `(:name "Sections"
    :category location
    :action ,#'consult-cabal--section-action
    :items ,#'consult-cabal--section-items)
  "Definition of source for Cabal sections.")

which means I need two functions, consult-cabal--section-action and consult-cabal--section-items. I started with the latter.

Getting section nodes as items for consult

It took me a while to work understand how this would ever be able to work. The function that :items point to must return a list of strings, but how would I ever be able to use just a string to jump to the correct location?

The solution is in a comment in the documentation of consult--multi:

:items - List of strings to select from or function returning list of strings. Note that the strings can use text properties to carry metadata, which is then available to the :annotate, :action and :state functions.

I'd never come across text properties in Emacs before, so at first I completely missed those two words. Once I'd looked up the concept in the documentation everything fell into place. The function consult-cabal--section-items would simply attach the relevant node as a text property to the strings in the list.

My current version, obviously a work-in-progress, takes a list of nodes and turns them naïvely into a string and attaches the node. I split it into two functions, like this

(defun consult-cabal--section-to-string (section)
  "Convert a single SECTION node to a string."
  (propertize (format "%S" section)
              :treesit-node section))

(defun consult-cabal--section-items ()
  "Fetch all sections as a list of strings ."
  (let ((section-nodes (treesit-query-capture (treesit-buffer-root-node 'cabal)
                                              "(cabal (sections (_)* @section))"
                                              nil nil t)))
    (mapcar #'consult-cabal--section-to-string section-nodes)))

Implementing the action

The action function is called with the selected item, i.e. with the string and its properties. That means, to jump to the selected section the function needs to extract the node property, :treesit-node, and jump to the start of it. the function to use is get-text-property, and as all characters in the string will have to property I just picked the first one. The jumping itself I copied from the navigation functions I'd written before.

(defun consult-cabal--section-action (item)
  "Go to the section referenced by ITEM."
  (when-let* ((node (get-text-property 0 :treesit-node item))
              (new-pos (treesit-node-start node)))
    (goto-char new-pos)))

Tying it together with consult--multi

The final function, consult-cabal, looks like this

(defun consult-cabal ()
  "Choose a Cabal construct and jump to it."
  (interactive)
  (consult--multi '(consult-cabal--source-section)
                  :sort nil))

Conclusions and where to find the code

The end result works as intended, but it's very rough. I'll try to improve it a bit more. In particular I want

  1. better strings - (format "%S" node) is all right to start with, but in the long run I want strings that describe the sections, and
  2. preview as I navigate between items - AFAIU this is what the :state field is for, but I still haven't looked into how it works.

The source can be found here.

Tags: cabal consult emacs tree-sitter
Other posts