Posts tagged "emacs":

19 Nov 2023

Making Emacs without terminal emulator a little more usable

After reading Andrey Listopadov's You don't need a terminal emulator (mentioned at Irreal too) I decided to give up on using Emacs as a terminal for my shell. In my experience Emacs simply isn't a very good terminal to run a shell in anyway. I removed the almost completely unused shell-pop from my configuration and the keybinding with a binding to async-shell-command. I'm keeping terminal-here in my config for the time being though.

I realised projectile didn't have a function for running it in the root of a project, so I wrote one heavily based on project-async-shell-command.

(defun mep-projectile-async-shell-command ()
  "Run `async-shell-command' in the current project's root directory."
  (declare (interactive-only async-shell-command))
  (let ((default-directory (projectile-project-root)))
    (call-interactively #'async-shell-command)))

I quickly found that the completion offered by Emacs for shell-command and async-shell-command is far from as sophisticated as what I'm used to from Z shell. After a bit of searching I found emacs-bash-completion. Bash isn't my shell of choice, partly because I've found the completion to not be as good as in Z shell, but it's an improvement over what stock Emacs offers. The instructions in the repo was good, but had to be adjusted slightly:

(use-package bash-completion
  :straight (:host github
             :repo "szermatt/emacs-bash-completion")
  (add-hook 'shell-dynamic-complete-functions 'bash-completion-dynamic-complete))

I just wish I'll find a package offering completions reaching Z shell levels.

Tags: emacs
16 Nov 2023

Using the golang mode shipped with Emacs

A few weeks ago I wanted to try out tree-sitter and switched a few of the modes I use for coding to their -ts-mode variants. Based on the excellent How to Get Started with Tree-Sitter I added bits like this to the setup I have for coding modes:1

(use-package X-mode
  (add-to-list 'treesit-language-source-alist '(X ""))
  ;; (treesit-install-language-grammar 'X)
  (add-to-list 'major-mode-remap-alist '(X-mode . X-ts-mode))
  ;; ...

I then manually evaluated the expression that's commented out to download and compile the tree-sitter grammar. It's a rather small change, it works, and I can switch over language by language. I swapped a couple of languages to the tree-sitter modes like this, including golang. The only mode that I noticed changes in was golang, in particular my adding of gofmt-before-save to before-save-hook had stopped having any effect.

What I hadn't realised was that the go-mode I was using didn't ship with Emacs and that when I switched to go-ts-mode I switched to one that was. It turns out that gofmt-before-save is hard-wired to work only in go-mode, something others have noticed.

I don't feel like waiting for go-mode to fix that though, especially not when there's a perfectly fine golang mode shipping with Emacs now, and not when emacs-reformatter make it so easy to define formatters (as I've written about before).

My golang setup, sans keybindings, now looks like this:2

(use-package go-ts-mode
  (go-ts-mode . lsp-deferred)
  (go-ts-mode . go-format-on-save-mode)
  (add-to-list 'treesit-language-source-alist '(go ""))
  (add-to-list 'treesit-language-source-alist '(gomod ""))
  ;; (dolist (lang '(go gomod)) (treesit-install-language-grammar lang))
  (add-to-list 'auto-mode-alist '("\\.go\\'" . go-ts-mode))
  (add-to-list 'auto-mode-alist '("/go\\.mod\\'" . go-mod-ts-mode))
  (reformatter-define go-format
    :program "goimports"
    :args '("/dev/stdin"))
  ;; ...

So far I'm happy with the built-in go-ts-mode and I've got to say that using a minor mode for the format-on-save functionality is more elegant than adding a function to before-save-hook (something that go-mode may get through this PR).



There were a few more things that I needed to modify. As the tree-sitter modes are completely separate from the non-tree-sitter modes things like hooks and keybindings in the modes' keymaps.


The full file is here.

Tags: emacs tree-sitter
01 Oct 2023

How I use Emacs

I've recently written two posts about my attempts to use a slimmed down Emacs setup for some very specific use cases. I'be put both posts on Reddit, here and here, and in both cases the majority of comments have been telling me that I should use emacsclient. I know they have good intentions – they want to share insight they've gained and benefit from on a daily basis. However, no matter how many Emacs devotees point out the benefits of emacsclient I'm not about to start using it. This post is an attempt to answer why that is.

Up front I want to clarify a few things:

  1. Yes, I know how to use emacsclient, and
  2. yes, I know it is a good way to, in a way, improve Emacs startup time,1 and
  3. yes, I know how to turn on server-mode in Emacs, and finally
  4. yes, I know how to run Emacs using a user unit for SystemD.

With that out of the way, here are the two ways I use Emacs

  1. As my starting point for work, i.e. writing code, keeping notes, tracking time, and writing my daily work journal.
  2. As my editor of ephemeral files.2

The next two sections explain more about these two distinct ways I use Emacs

As my starting point for work

Number of packages 162
Init time (emacs-init-time) 1.883483s
Config size (by du -bch) 68K

Much of what I do on a daily basis starts in Emacs. I typically have one instance of Emacs open and I always keep it on the second virtual desktop. I always run it in the GUI. When I write code I start with opening a new tab, then I open a dired buffer in the project's folder (by using consult-projectile). When I need a terminal I open it from Emacs using on of terminal-here-project-launch or terminal-here. Occasionally I open a shell prompt inside Emacs using shell-pop.

Back when I used Vim as my main editor I always started a terminal first and then opened files from there. Since switching to Emacs I've completely stopped doing that. Over the last 8 or so years of Emacs usage there's only been a handful of times when I've wanted to open a file from the terminal and I've run M-x server-start and used emacsclient. The last time was more than a year ago.

As I typically keep exactly one Emacs open, and I start it soon after logging in, I'm not too concerned with startup time. I think under 2s is more than fast enough given the functionality I have in my setup.

This setup I use for taking notes and writing my daily work journal, as well as reading email, and programming in a half-dozen languages. I have a large-ish set of keybindings that I've set up using general.el, inspired by Spacemacs at first but by now it's started to gain its own character.

As my editor of ephemeral files ($EDITOR)

Number of packages 22
Init time (emacs-init-time) 0.209298s
Config size (by du -bch) 6.7K

Ephemeral files are files I tend to edit for less than 30 seconds, maybe a minute at most. There are three main use cases for ephemeral files:

  1. Searching the scrollback buffer in zellij, and copying bits to the clipboard for various uses.
  2. Editing files when running git from the command line. It's not something I do very often, but it happens.
  3. Editing shell commands. When they get a little too large to handle conveniently using ZSH directly I invoke edit-command-line.

For a few reasons I decided to make a second, completely separate configuration just to handle ephemeral files.

  • It will only be used in a terminal.
  • I want to be able to have some special keybindings that suit a specific use, e.g. for the scrollback buffer I've bound `SPC Y` to copy the selected text to the clipboard and then exit Emacs. It's a thing that I use all the time with the scrollback buffer, but never otherwise.
  • I have no need, nor any desire, to switch from editing a commit message, or searching a scrollback buffer, to reading email or editing an org-mode file. The complete separation is a feature.

With a startup time of less than a quarter of a second it is well within the acceptable, and there is absolutely no need to use emacsclient just to speed things up. Given my desire for separation, I wouldn't want to use my main Emacs instance as a server and edit ephemeral files anyway.


I've found a setup that seems to work really well and tick all the requirements I have when it comes to separation between use cases and ability to have custom keybindings for them. Also, Emacs is starting up very fast with my slimmed down configuration. If starting Emacs with the slimmed configuration starts taking too long I'm more likely to go back to using Neovim than complicate things with emacsclient.

So no, I am not going to start using emacsclient any time soon.



I write "in a way" as it actually does nothing for Emacs startup time, it just shifts it to a point in time so you don't have to sit and wait for it to start.


I used to use Neovim, without any config, for most of this until recently.

Tags: emacs
30 Sep 2023

Using Emacs as $EDITOR

Continuing on from my experiment with using Emacs as for scrollback in my terminal multiplexer I thought I'd try to use it as my $EDITOR as well.

The two main cases where I use $EDITOR is

  1. The occasional use of git on the command line, rebasing or writing a commit message, and
  2. Use of ZSH's edit-command-line functionality.

To make sure Emacs is starting up quickly enough I'm using the same small setup I created for the scrollback editing, so I'm now setting EDITOR like this

export EDITOR="emacs -nw --init-directory ~/.se.d"

Now that I want to use the same setup for editing I can't really jump into view-mode every time Emacs starts so I have to be a bit more clever. The following bit won't do

(add-hook 'find-file-hook #'view-mode)

I need to somehow find out what starts Emacs and then only modify the hook when needed. Unfortunately I haven't found anything that reveals that Emacs is started by zellij. Creating a separate little script that zellij uses would be an option, of course, but for now I've opted to make it the default and instead refrain from adding the hook in the other two use cases.

ZSH doesn't make it easy to find out that it's edit-command-line either, but as I've observed that the command line sometimes doesn't look right after leaving the editor I wanted to call redisplay to fix it up. That means I need to have a function anyway, so using an environment variable becomes an easy way to check if Emacs is being used to edit the command line.

function se-edit-command-line() {
    export SE_SKIP_VIEW=y
    zle edit-command-line
    unset SE_SKIP_VIEW
    zle redisplay
zle -N se-edit-command-line

bindkey -M vicmd '^V' se-edit-command-line
bindkey -M viins '^V' se-edit-command-line

Unfortunately is seems zle edit-command-line doesn't pass on non-exported environment variables, hence the explicit export and unset.

When git starts an editor it sets a few environment variables so it was easy to just pick one that is set in both cases I care about. I picked GIT_EXEC_PATH.

With these things in place I changed the slim setup to only add the hook when neither of the environment variables are present

(unless (or (getenv "SE_SKIP_VIEW")
            (getenv "GIT_EXEC_PATH"))
  (add-hook 'find-file-hook #'view-mode))

Hopefully this works out well enough that I won't feel a need to go back to using Neovim as my $EDITOR.

Tags: emacs zsh
24 Sep 2023

Defining a formatter for Cabal files

For Haskell code I can use lsp-format-buffer and lsp-format-region to keep my file looking nice, but I've never found a function for doing the same for Cabal files. There's a nice command line tool, cabal-fmt, for doing it, but it means having to jump to a terminal. It would of course be nicer to satisfy my needs for aesthetics directly from Emacs. A few times I've thought of writing the function myself, I mean how hard can it be? But then I've forgotten about it until then next time I'm editing a Cabal file.

A few days ago I noticed emacs-reformatter popping up in my feeds. That removed all reasons to procrastinate. It turned out to be very easy to set up.

The package doesn't have a recipe for straight.el so it needs a :straight section. Also, the naming of the file in the package doesn't fit the package name, hence the slightly different name in the use-package declaration:1

(use-package reformatter
  :straight (:host github
             :repo "purcell/emacs-reformatter"))

Now the formatter can be defined

(reformatter-define cabal-format
  :program "cabal-fmt"
  :args '("/dev/stdin"))

in order to create functions for formatting, cabal-format-buffer and cabal-format-region, as well as a minor mode for formatting on saving a Cabal file.



I'm sure it's possible to use :files to deal with this, but I'm not sure how and my naive guess failed. It's OK to be like this until I figure it out properly.

Tags: emacs haskell
16 Sep 2023

Setting up emacs-openai/chatgpt

Yesterday I decided to try to make more use of the ChatGPT account I have. What prompted it mostly was that I recalled that my employer has a paid subscription and that if we use it enough they'll get us access to ChatGPT4.

After a bit of research1 I decided to start with emacs-open/chatgpg. However, as I found the instructions slightly lacking I'm sharing my setup.

The instructions for straight.el fail to mention that one needs the openai package too.

(use-package openai
  :straight (openai :type git :host github :repo "emacs-openai/openai"))

The complete declaration for use-package ended up looking like this:

(use-package chatgpt
  :straight (chatgpt :type git :host github :repo "emacs-openai/chatgpt")
  :requires openai
  (setq openai-key #'openai-key-auth-source))

Oh, and don't forget to put an entry into `~/.authinfo.gpg`. Something like this should do it

machine login <anything> password <your key>



I found Alex Kehayias' note to be a good starting point.

Tags: emacs openai chatgpt
10 Sep 2023

Using emacs for the scrollback in terminal multiplexers

I should start with saying that I still don't really know if this is a good idea or not, but it feels like it's worth trying out at least.

An irritating limitation in Zellij, and a possible solution

After seeing it mentioned in an online community I thought it might be worth trying out. I'm not really disappointed with tmux, I've been using it for years but I actually only use a small part of what it can do. I create tabs, sometimes create panes, and I regularly use the scollback functionality to copy output of commands I've run earlier.

Unfortunately, as is reported in a ticket, Zellij can't select and copy using the keyboard. From the discussion in that ticket it seems unlikely it ever will be able to. After finding that out I resigned to staying with Tmux – I'm not ready to go back to using a pointing device to select and copy text in my terminal!

When I was biking to the pool yesterday I realised a thing though: I'm already using a tool that is very good at manipulating text using the keyboard. Of course I'm talking about Emacs! So if I can just make Emacs start up quickly enough I ought to be able to use it for searching, selecting and copying from the scrollback buffer. I haven't found a way to do this in Tmu= yet, but Zellij has EditScrollBack so I can at least try it out.

A nice benefit is that I can cut back on the number of different shortcuts I use daily.

My slimmed down Emacs config

My current Emacs config starts up in less than 2s, which is good enough as I normally start Emacs at most a few times per day. However, if I have to wait 2s to open the scrollback buffer I suspect I'll tire very quickly and abandon the experiment. So I took my ordinary config and slimmed it down. Cutting away things that weren't related to navigation and searching.

The list of packages is not very long:

  • straight
  • use-package
  • evil
  • general
  • which-key
  • vertico
  • orderless
  • marginalia
  • consult

The first 5 are for usability in general, and the last 4 are bring in the functions I use for searching through text.

The resulting config starts in less than ¼s. That's more than acceptable, I find.

Copying to the Wayland clipboard

It turns out that copying with `y` in evil does the right thing by default and the copied text ends up in the clipboard without any special configuration.

From Tmux I'm used to being thrown out of copy-mode after copying something. While that's sometimes irritating, it's at other times exactly what I want. Given that evil-yank does the right thing it was easy to write a function for it:

(defun se/yank-n-kill (beg end)
  (interactive "r")
  (evil-yank beg end)

The config files

I'm keeping my dot-files in a private repo, but I put a snapshot of the Emacs and Zellij config in a snippet.

Tags: emacs tmux zellij
26 Jul 2023

Making keymaps prettier with general.el

After my previous post on defining keymaps using general.el I revisited some of my setup in order to make some package keymaps prettier when displayed by which-key. The keymaps defined by packages are typically don't contain any description of the bindings, so which-key ends up displaying the name of the function bound to the key. It's not always easy to remember what a function does based just on its name, and sometimes the names are so long that which-key cuts the name short and all you see is a bunch of keybindings for seemingly the same function.

The only remedy I've found is to re-define the keybindings decorating them with descriptions that (hopefully) are easier to understand than the function name. (At least, this way I only have myself to blame if it's a bad description.)

One such keymap that I use somewhat frequently, and where the function names often confuse me is evil-mc-key-map from evil-mc. The keymap is bound at g . in evil's normal and visual modes, and it also contains a few bindings using control (C-) and meta (M-) which I think would be better placed under sub-keymaps.

I decided to group the "make and go" functions under g . m and the "skip and go" function under g . s. The rest I'm just giving descriptions. general.el makes it easy both to overwrite already existing bindings, but adding descriptions, and to add new ones, all in one call to general-def:

(general-def evil-mc-key-map
  :states '(normal visual)
  "g.A" '("make end sel" . evil-mc-make-cursor-in-visual-selection-end)
  "g.I" '("make beg sel" . evil-mc-make-cursor-in-visual-selection-beg)
  "g.a" '("make all" . evil-mc-make-all-cursors)
  "g.q" '("undo all" . evil-mc-undo-all-cursors)
  "g.u" '("undo last" . evil-mc-undo-last-added-cursor)
  "g. RET" '("make here" . evil-mc-make-cursor-here)
  "g.p" '("pause" . evil-mc-pause-cursors)
  "g.r" '("resume" . evil-mc-resume-cursors)

  "g.m" '(:ignore t :wk "make & go")
  "g.m$" '("to last cur" . evil-mc-make-and-goto-last-cursor)
  "g.m0" '("to first cur" . evil-mc-make-and-goto-first-cursor)
  "g.mC" '("to prev cur" . evil-mc-make-and-goto-prev-cursor)
  "" '("to next cur" . evil-mc-make-and-goto-next-cursor)
  "" '("to prev match" . evil-mc-make-and-goto-prev-match)
  "g.mj" '("to next line" . evil-mc-make-cursor-move-next-line)
  "" '("to prev line" . evil-mc-make-cursor-move-prev-line)
  "" '("to next match" . evil-mc-make-and-goto-next-match)

  "g.s" '(:ignore t :wk "skip & go")
  "g.sC" '("to prev cur" . evil-mc-skip-and-goto-prev-cursor)
  "" '("to next cur" . evil-mc-skip-and-goto-next-cursor)
  "" '("to prev match" . evil-mc-skip-and-goto-prev-match)
  "" '("to next match" . evil-mc-skip-and-goto-next-match))

Finally I want to remove the bindings that weren't overwritten. general.el makes that easy too with general-undbind:

(general-unbind '(normal visual) evil-mc-key-map
  "g.$" "g.0"
  "g. C-n" "g. C-S-n"
  "g. C-u" "g. C-S-u"
  "g. C-p" "g. C-r"
  "g. M-N" "g. M-n"
  "g.N" "g.O" "g.n" "g.o")

Yes, this is a bit of work, and so far I've only done this for a few keymaps (those containing bindings I use frequently and/or find difficult to remember). So far I've found it worth the cost.

Tags: emacs general.el
09 Jul 2023

general.el and two ways to define keybindings

When I abandoned spacemacs I really wanted to duplicate its keybindings using SPC as leader key and per-mode bindings available by pressing ,. I found a nice setup using general.el in Tianshu Wang's Emacs config. I made only minor modification and ended up with the following setup.

(use-package general
  :after (evil evil-easymotion)

   :states '(normal insert motion visual emacs)
   :keymaps 'override
   :prefix-map 'tyrant-map
   :prefix "SPC"
   :non-normal-prefix "M-SPC")
  (general-create-definer mes/tyrant-def :keymaps 'tyrant-map)
  (mes/tyrant-def "" nil)

  (general-create-definer mes/despot-def
    :states '(normal insert motion visual emacs)
    :keymaps 'override
    :major-modes t
    :prefix ","
    :non-normal-prefix "M-,")
  (mes/despot-def "" nil)

  (general-def universal-argument-map
    "SPC u" 'universal-argument-more))

One slightly surprising thing I found out is that two different ways to define keybindings can be used, one seems to work on both top and mode level, the other only on mode level.

Top-level keybindings (SPC)

At the top-level, i.e. when using mes/tyrant-def, I need to use a cons based configuration. This is part of my top-level keybindings:

  "SPC" '("M-x" . execute-extended-command)
  "TAB" '("latest buffer" . mode-line-other-buffer)
  "!" 'shell-command
  "/" '("search" . consult-ripgrep)
  "u" 'universal-argument

  "b" (cons "bufs" (make-sparse-keymap))
  "bb" '("switch" . consult-buffer)
  "bc" '("close" . kill-this-buffer)
  "be" '("erase" . erase-buffer)
  "bs" '("scratch" . scratch-buffer)

  ;; ....

When it's done this way which-key picks up the descriptive strings.

It did take me a while to figure out that this config style was necessary. Again Tianshu Wang's Emacs config was what led me right.

Mode-level keybindings (,)

At the mode-level, i.e. using mes/despot-def, I've found that I can, almost always, define keybindings following the documentation of general.el. Here's the keybindings I have for nix-mode:

(mes/despot-def nix-mode-map
  "=" '(:ignore t :wk "format")
  "=b" '(nix-format-buffer :wk "buffer"))

Just the other day I found that AUCTeX must be doing something with it's keymaps and I have to use the cons-based configuration style for its keymaps.

Tags: emacs general.el
30 Mar 2023

More on tree-sitter and consult

Here's a few things that I've gotten help with figuring out during the last few days. Both things are related to my playing with tree-sitter that I've written about earlier, here and here.

You might also be interested in the two repositories where the full code is. (I've linked to the specific commits as of this writing.)

Anonymous nodes and matching in tree-sitter

In the grammar for Cabal I have a rule for sections that like this

sections: $ => repeat1(choice(

where each section followed this pattern

benchmark: $ => seq(
    field('name', $.section_name),
    field('properties', $.property_block),

This made it a little bit difficult to capture the relevant parts of each section to implement consult-cabal. I thought a pattern like this ought to work

  (_ _ @type
     name: (section_name)? @name)))

but it didn't; I got way too many things captured in type. Clearly I had misunderstood something about the wildcards, or the query syntax. I attempted to add a field name to the anonymous node, i.e. change the sections rules like this

benchmark: $ => seq(
    field('type', 'benchmark'),
    field('name', $.section_name),
    field('properties', $.property_block),

It was accepted by tree-sitter generate, but the field type was nowhere to be found in the parse tree.

Then I changed the query to list the anonymous nodes explicitly, like this

  (_ ["benchmark" "common" "executable" ...] @type
     name: (section_name)? @name)))

That worked, but listing all the sections like that in the query didn't sit right with me.

Luckily there's a discussions area in tree-sitters GitHub so a fairly short discussion later I had answers to why my query behaved like it did and a solution that would allow me to not list all the section types in the query. The trick is to wrap the string in a call to alias to make it a named node. After that it works to add a field name to it as well, of course. The section rules now look like this

benchmark: $ => seq(
    field('type', alias('benchmark', $.section_type)),
    field('name', $.section_name),
    field('properties', $.property_block),

and the final query looks like this

   type: (section_type) @type
   name: (section_name)? @name)))

With that in place I could improve on the function that collects all the items for consult-cabal so it now show the section's type and name instead of the string representation of the tree-sitter node.

State in a consult source for preview of lines in a buffer

I was struggling with figuring out how to make a good state function in order to preview the items in consult-cabal. The GitHub repo for consult doesn't have discussions enabled, but after a discussion in an issue I'd arrived at a state function that works very well.

The state function makes use of functions in consult and looks like this

(defun consult-cabal--state ()
  "Create a state function for previewing sections."
  (let ((state (consult--jump-state)))
    (lambda (action cand)
      (when cand
        (let ((pos (get-text-property 0 'section-pos cand)))
          (funcall state action pos))))))

The trick here was to figure out how the function returned by consult--jump-state actually works. On the surface it looks like it takes an action and a candidate, (lambda (action cand) ...). However, the argument cand shouldn't be the currently selected item, but rather a postion (ideally a marker), so I had to attach another text property on the items (section-pos, which is fetched in the inner lambda). This position is then what's passed to the function returned by consult--jump-state.

In hindsight it seems so easy, but I was struggling with this for an entire evening before finally asking the question the morning after.

Tags: consult emacs tree-sitter
27 Mar 2023

Cabal, tree-sitter, and consult

After my last post I thought I'd move on to implement the rest of the functions in haskell-mode's major mode for Cabal, functions like haskell-cabal-goto-library-section and haskell-cabal-goto-executable-section. Then I realised that what I really want is a way to quickly jump to any section, that is, I want consult-cabal!

What follows is very much a work-in-progress, but hopefully it'll show enough promise.

Listing the sections

As I have a tree-sitter parse tree to hand it is fairly easy to fetch all the nodes corresponding to sections. Since the last post I've made some improvements to the parser and now the parse tree looks like this (I can recommend the function treesit-explore-mode to expore the parse tree, I've found it invaluable ever since I realised it existed)

 (properties ...)
  (common common (section_name) ...)
  (library library ...)
  (executable executable (section_name) ...)

That is, all the sections are children of the node called sections.

The function to use for fetching all the nodes is treesit-query-capture, it needs a node to start on, which this case should be the full parse tree, i.e. (treesit-buffer-root-node 'cabal) and a query string. Given the structure of the parse tree, and that I want to capture all children of sections, a query string like this one works

"(cabal (sections (_)* @section))"

Finally, by default treesit-query-capture returns a list of tuples of the form (<capture> . <node>), but in this case I only want the list of nodes, so the full call will look like this

(treesit-query-capture (treesit-buffer-root-node 'cabal)
                       "(cabal (sections (_)* @section))"
                       nil nil t)

Hooking it up to consult

As I envision adding more things to jump to in the future, I decided to make use of consult--multi. That in turn means I need to define a "source" for the sections. After a bit of digging and rummaging in the consult source I put together this

(defvar consult-cabal--source-section
  `(:name "Sections"
    :category location
    :action ,#'consult-cabal--section-action
    :items ,#'consult-cabal--section-items)
  "Definition of source for Cabal sections.")

which means I need two functions, consult-cabal--section-action and consult-cabal--section-items. I started with the latter.

Getting section nodes as items for consult

It took me a while to work understand how this would ever be able to work. The function that :items point to must return a list of strings, but how would I ever be able to use just a string to jump to the correct location?

The solution is in a comment in the documentation of consult--multi:

:items - List of strings to select from or function returning list of strings. Note that the strings can use text properties to carry metadata, which is then available to the :annotate, :action and :state functions.

I'd never come across text properties in Emacs before, so at first I completely missed those two words. Once I'd looked up the concept in the documentation everything fell into place. The function consult-cabal--section-items would simply attach the relevant node as a text property to the strings in the list.

My current version, obviously a work-in-progress, takes a list of nodes and turns them naïvely into a string and attaches the node. I split it into two functions, like this

(defun consult-cabal--section-to-string (section)
  "Convert a single SECTION node to a string."
  (propertize (format "%S" section)
              :treesit-node section))

(defun consult-cabal--section-items ()
  "Fetch all sections as a list of strings ."
  (let ((section-nodes (treesit-query-capture (treesit-buffer-root-node 'cabal)
                                              "(cabal (sections (_)* @section))"
                                              nil nil t)))
    (mapcar #'consult-cabal--section-to-string section-nodes)))

Implementing the action

The action function is called with the selected item, i.e. with the string and its properties. That means, to jump to the selected section the function needs to extract the node property, :treesit-node, and jump to the start of it. the function to use is get-text-property, and as all characters in the string will have to property I just picked the first one. The jumping itself I copied from the navigation functions I'd written before.

(defun consult-cabal--section-action (item)
  "Go to the section referenced by ITEM."
  (when-let* ((node (get-text-property 0 :treesit-node item))
              (new-pos (treesit-node-start node)))
    (goto-char new-pos)))

Tying it together with consult--multi

The final function, consult-cabal, looks like this

(defun consult-cabal ()
  "Choose a Cabal construct and jump to it."
  (consult--multi '(consult-cabal--source-section)
                  :sort nil))

Conclusions and where to find the code

The end result works as intended, but it's very rough. I'll try to improve it a bit more. In particular I want

  1. better strings - (format "%S" node) is all right to start with, but in the long run I want strings that describe the sections, and
  2. preview as I navigate between items - AFAIU this is what the :state field is for, but I still haven't looked into how it works.

The source can be found here.

Tags: cabal consult emacs tree-sitter
22 Mar 2023

Making an Emacs major mode for Cabal using tree-sitter

A few days ago I posted on r/haskell that I'm attempting to put together a Cabal grammar for tree-sitter. Some things are still missing, but it covers enough to start doing what I initially intended: experiment with writing an alternative Emacs major mode for Cabal.

The documentation for the tree-sitter integration is very nice, and several of the major modes already have tree-sitter variants, called X-ts-mode where X is e.g. python, so putting together the beginning of a major mode wasn't too much work.

Configuring Emacs

First off I had to make sure the parser for Cabal was installed. The snippet for that looks like this1

(use-package treesit
  :straight nil
  :ensure nil
  :commands (treesit-install-language-grammar)
  (setq treesit-language-source-alist
        '((cabal . ("")))))

With that in place the parser is installed using M-x treesit-install-language-grammar and choosing cabal.

After that I removed my configuration for haskell-mode and added the following snippet to get my own major mode into my setup.

(use-package my-cabal-mode
  :straight (:type git
             :repo ""
             :branch "main"
             :files (:defaults "my-cabal-mode/*el")))

The major mode and font-locking

The built-in elisp documentation actually has a section on writing a major mode with tree-sitter, so it was easy to get started. Setting up the font-locking took a bit of trial-and-error, but once I had comments looking the way I wanted it was easy to add to the setup. Oh, and yes, there's a section on font-locking with tree-sitter in the documentation too. At the moment it looks like this

(defvar cabal--treesit-font-lock-setting
   :feature 'comment
   :language 'cabal
   '((comment) @font-lock-comment-face)

   :feature 'cabal-version
   :language 'cabal
   '((cabal_version _) @font-lock-constant-face)

   :feature 'field-name
   :language 'cabal
   '((field_name) @font-lock-keyword-face)

   :feature 'section-name
   :language 'cabal
   '((section_name) @font-lock-variable-name-face))
  "Tree-sitter font-lock settings.")

(define-derived-mode my-cabal-mode fundamental-mode "My Cabal"
  "My mode for Cabal files"

  (when (treesit-ready-p 'cabal)
    (treesit-parser-create 'cabal)
    ;; set up treesit
    (setq-local treesit-font-lock-feature-list
                '((comment field-name section-name)
                  () ()))
    (setq-local treesit-font-lock-settings cabal--treesit-font-lock-setting)

(add-to-list 'auto-mode-alist '("\\.cabal\\'" . my-cabal-mode))


One of the reasons I want to experiment with tree-sitter is to use it for code navigation. My first attempt is to translate haskell-cabal-section-beginning (in haskell-mode, the source) to using tree-sitter. First a convenience function to recognise if a node is a section or not

(defun cabal--node-is-section-p (n)
  "Predicate to check if treesit node N is a Cabal section."
  (member (treesit-node-type n)
          '("benchmark" "common" "executable" "flag" "library" "test_suite")))

That makes it possible to use treesit-parent-until to traverse the nodes until hitting a section node

(defun cabal-goto-beginning-of-section ()
  "Go to the beginning of the current section."
  (when-let* ((node-at-point (treesit-node-at (point)))
              (section-node (treesit-parent-until node-at-point #'cabal--node-is-section-p))
              (start-pos (treesit-node-start section-node)))
    (goto-char start-pos)))

And the companion function, to go to the end of a section is very similar

(defun cabal-goto-end-of-section ()
  "Go to the end of the current section."
  (when-let* ((node-at-point (treesit-node-at (point)))
              (section-node (treesit-parent-until node-at-point #'cabal--node-is-section-p))
              (end-pos (treesit-node-end section-node)))
    (goto-char end-pos)))



I'm using straight.el and use-package in my setup, but hopefully the snippets can easily be converted to other ways of configuring Emacs.

Tags: cabal emacs haskell tree-sitter
03 Mar 2023

Per-project xref history in Emacs

When I write code I jump around in the code quite a bit, as I'm sure many other developers do. The ability to jump to the definition of a function, or a type, is invaluable when trying to understand code. In Emacs the built-in xref package provides the basic functionality for this, with many other packages providing their custom functions for looking up identifiers. This works beautifully except for one thing, there's only one global stack for keeping track of how you've jumped around.

Well, that used to be the case.

As I tend to have multiple projects open at a time I used to find it very confusing when I pop positions off the xref stack and all of a sudden find myself in another project. It would be so much nicer to have a per-project stack.

I've only known of one solution for this, the perspective package, but as I've been building my own Emacs config I wanted to see if there were other options. It turns out there is one (almost) built into Emacs 29.

In Emacs 29 there's built-in support for having per-window xref stacks, and the way that's done allows one to extend it further. There's now a variable, xref-history-storage, that controls access to the xref stack. The default is still a global stack (when the variable is set to #'xref-global-history), but to get per-window stacks one sets it to #'xref-window-local-history.

After finding this out I thought I'd try to write my own, implementing per-project xref stacks (for projectile).

The function should take one optional argument, new-value, if it's provided the stack should be updated and if not, it should be returned. That is, something like this

(defun projectile-param-xref-history (&optional new-value)
  "Return project-local xref history for the current projectile.

Override existing value with NEW-VALUE if it's set."
  (if new-value
      (projectile-param-set-parameter 'xref--history new-value)
    (or (projectile-param-get-parameter 'xref--history)
        (projectile-param-set-parameter 'xref--history (xref--make-xref-history)))))

Now I only had to write the two functions projectile-param-get-parameter and projectile-param-set-parameter. I thought a rather straight forward option would be to use a hashtable and store values under a tuple comprising the project name and the parameter passed in.

(defvar projectile-params--store (make-hash-table :test 'equal)
  "The store of project parameters.")

(defun projectile-param-get-parameter (param)
  "Return project parameter PARAM, or nil if unset."
  (let ((key (cons (projectile-project-name) param)))
    (gethash key projectile-params--store nil)))

(defun projectile-param-set-parameter (param value)
  "Set the project parameter PARAM to VALUE."
  (let ((key (cons (projectile-project-name) param)))
    (puthash key value projectile-params--store))

Then I tried it out by setting xref-history-storage

(setq xref-history-storage #'projectile-param-xref-history)

and so far it's been working well.

The full code is here.

Tags: emacs xref projectile
24 Sep 2022

Annotate projects in Emacs

Every now and then I've wished to write comments on files in a project, but I've never found a good way to do that. annotate.el and org-annotate-file both collect annotations in a central place (in my $HOME), while marginalia puts annotations in files next to the source files but in a format that's rather cryptic and tends to be messed up when attached to multiple lines. None of them is ideal, I'd like the format to be org-mode, but not in a central file. At the same time having one annotation file per source file is simply too much.

I tried wrapping org-annotate-file, setting org-annotate-file-storage-file and taking advantage of elisp's dynamic binding. However, it opens the annotation file in the current window, and I'd really like to split the window and open the annotations the right. Rather than trying to sort of "work it out backwards" I decided to write a small package and use as much of the functionality in org-annotate-file.el as possible.

First off I decided that I want the annotation file to be called

(defvar org-projectile-annotate-file-name ""
  "The name of the file to store project annotations.")

Then I wanted a slightly modified version of org-annotate-file-show-section, I wanted it to respect the root of the project.

(defun org-projectile-annotate--file-show-section (storage-file)
  "Add or show annotation entry in STORAGE-FILE and return the buffer."
  ;; modified version of org-annotate-file-show-section
  (let* ((proj-root (projectile-project-root))
         (filename (file-relative-name buffer-file-name proj-root))
         (line (buffer-substring-no-properties (point-at-bol) (point-at-eol)))
         (annotation-buffer (find-file-noselect storage-file)))
    (with-current-buffer annotation-buffer
      (org-annotate-file-annotate filename line))

The main function can then simply work out where the file with annotations should be located and call org-projectile-annotate--file-show-section.

(defun org-projectile-annotate ()
  (let ((annot-fn (file-name-concat (projectile-project-root)
    (set-window-buffer (split-window-right)
                       (org-projectile-annotate--file-show-section annot-fn))))

When testing it all out I noticed that org-store-link makes a link with a search text. In my case it would be much better to have links with line numbers. I found there's a hook to modify the behaviour of org-store-link, org-create-file-search-functions. So I wrote a function to get the kind of links I want, but only when the project annotation file is open in a buffer.

(defun org-projectile-annotate-file-search-func ()
  "A function returning the current line number when called in a
project while the project annotation file is open.

This function is designed for use in the hook
'org-create-file-search-functions'. It changes the behaviour of
'org-store-link' so it constructs a link with a line number
instead of a search string."
  ;; TODO: find a way to make the link description nicer
  (when (and (projectile-project-p)
             (get-buffer-window org-projectile-annotate-file-name))
    (number-to-string (line-number-at-pos))))

That's it, now I only have to wait until the next time I want to comment on a project to see if it improves my way of working.

Tags: emacs org-mode
09 Jul 2022

Playing with setting up Emacs

TL;DR: I've put together a small-ish Emacs configuration that I call MES. Hopefully it can be of use to someone out there.

My Emacs Setup - MES

The other day I started watching some videos in the Emacs From Scratch series from System Crafters. It looked like something that could be fun to play with so over the last couple of days I've been tinkering with putting together the beginnings of a configuration.

During the process I realised just how much work it'd be to put together something that comes close to the polish of Spacemacs, so I've currently no intention of actually using MES myself. It was fun though, and maybe it can serve as inspiration (or as a deterrent) for someone else.

The major parts are

Tags: emacs
15 Jun 2022

Power-mode in Spacemacs

I just found the Power Mode for Emacs. If you want to try it out in Spacemacs you can make sure that your ~/.spacemacs contains the following

  (power-mode :location (recipe
                         :fetcher github
                         :repo "elizagamedev/power-mode.el"))

After a restart Power Mode can be turned on using SPC SPC power-mode.

Unfortunately I found that it slows down rendering so badly that Emacs isn't keeping up with my typing. Even though I removed it right away again it was fun to try it out, and I did learn how to add package to Spacemacs that aren't on MELPA.

A useful resource is this reference on the recipe format.

Tags: emacs spacemacs
09 May 2022

Comments and org-static-blog

I'm using org-static-blog to generate the contents of this site. So far I'm very happy with it, but I've gotten a few emails from readers who've wanted to comment on something I've written and they always point out that it's not easy to do. It's actually not a coincidence that it's a bit difficult!

Yesterday I came up with a way that might make is slightly easier without involving JavaScript from a 3rd party. By making use of the built-in support for adding HTML code for comments. One slight limitation is that it's a single variable holding the code, and I'd really like to allow for both

As the comment support in org-static-blog comes in the form of a single variable this seems a bit difficult to accomplish. However, it isn't difficult at all to do in elisp due to the power of advice-add.

By using the following advice on org-static-blog-publish-file

(advice-add 'org-static-blog-publish-file :around
            (lambda (orig-fn filename &rest args)
              (let*  ((comments-url (with-temp-buffer
                                      (insert-file-contents filename)
                                      (or (cadar (org-collect-keywords '("commentsurl")))
                      (org-static-blog-post-comments (concat "Comment <a href=" comments-url ">here</a>.")))
                (apply orig-fn filename args))))

and defining my-blog-default-comments-url to a mailto:... URL I get a link to use for commenting by either

  1. set commentsurl to point to discussion about the post on reddit, or
  2. not set commentsurl at all and get the mailto:... URL.

If you look at my previous post you see the result of the former, and if you look below you see the result of the latter.

Tags: emacs org-mode
30 Jan 2022

Keeping Projectile's cache tidy

A while back I added a function to find all projects recursively from a directory and add them to Projectile's cache (see the Populating Projectile's cache). Since then I've just made a tiny change to also include containing a .projectile file.

(defun projectile-extra-add-projects-in-subfolders (projects-root)
  (interactive (list (read-directory-name "Add to known projects: ")))
  (message "Searching for projects in %s..." projects-root)
  (let* ((proj-rx (rx (and line-start ?. (or "projectile" "git") line-end)))
         (dirs (seq-map
                (directory-files-recursively projects-root
    (seq-do 'projectile-add-known-project dirs)
    (message "Added %d projects" (length dirs))))

Since then I've also found a need for tidying the cache, in my casse that means removing entries in the cache that no longer exist on disk. I didn't find a function for it, so I wrote one.

(defun projectile-extra-tidy-projects ()
  (let ((missing-dirs (seq-remove 'file-directory-p projectile-known-projects)))
    (seq-do 'projectile-remove-known-project missing-dirs)
    (message "Tidied %d projects" (length missing-dirs))))
Tags: emacs elisp projectile
01 Jan 2022

Trimming newline on code block variable

Today I found ob-http and decided to try it out a little. I quickly ran into a problem of a trailing newline. Basically I tried to do something like this:

#+name: id
#+begin_src http :select .id :cache yes
POST /foo
Content-Type: application/json

  "foo": "toto",
  "bar": "tata"

#+RESULTS[c5fd99206822a2109d7ac1d140185e6ec3f4f1d9]: id

#+header: :var id=id
#+begin_src http
POST /foo/${id}/fix

The trailing newline messes up the URL though, and the second code block fails.

I found two ways to deal with it, using a table and using org-sbe

Using a table

#+name: id
#+begin_src http :select .id :cache yes :results table
POST /foo
Content-Type: application/json

  "foo": "toto",
  "bar": "tata"

#+RESULTS[c5fd99206822a2109d7ac1d140185e6ec3f4f1d9]: id
| 48722051-f81b-433f-acb4-a65d961ec841 |

#+header: :var id=id[0,0]
#+begin_src http
POST /foo/${id}/fix

Using org-sbe

#+name: id
#+begin_src http :select .id :cache yes
POST /foo
Content-Type: application/json

  "foo": "toto",
  "bar": "tata"

#+RESULTS[c5fd99206822a2109d7ac1d140185e6ec3f4f1d9]: id

#+header: :var id=(org-sbe id)
#+begin_src http
POST /foo/${id}/fix
Tags: emacs org-mode
08 Dec 2021

Magit/forge and self-hosted GitLab

As I found the documentation for adding a self-hosted instance of GitLab to to magit/forge a bit difficult, I thought I'd write a note for my future self (and anyone else who might find it useful).

First put the following in `~/.gitconfig`

[gitlab ""]
  user = my.username

Then create an access token on GitLab. I ticked api and write_repository, which seems to work fine so far. Put the token in ~/.authinfo.gpg

machine login my.user^forge password <token>

(Remember that a newline is needed at the end of the file.)

Finally, add the GitLab instance to 'forge-alist

 '(("" "" "" forge-gitlab-repository)
   ("" "" "" forge-github-repository)
   ("" "" "" forge-gitlab-repository))

That's it!

Tags: emacs git magit
23 Jul 2021

Keeping todo items in org-roam v2

Org-roam v2 has been released and yes, it broke my config a bit. Unfortunately the v1-to-v2 upgrade wizard didn't work for me. I realized later that it might have been due to the roam-related functions I'd hooked into `'before-save-hook`. I didn't think about it until I'd already manually touched up almost all my files (there aren't that many) so I can't say anything for sure. However, I think it might be a good idea to keep hooks in mind if one runs into issues with upgrading.

The majority of the time I didn't spend on my notes though, but on the setup I've written about in an earlier post, Keeping todo items in org-roam. Due to some of the changes in v2, changes that I think make org-roam slightly more "org-y", that setup needed a bit of love.

The basis is still the same 4 functions I described in that post, only the details had to be changed.

I hope the following is useful, and as always I'm always happy to receive commends and suggestions for improvements.

Some tag helpers

The very handy functions for extracting tags as lists seem to be gone, in their place I found org-roam-{get,set}-keyword. Using these I wrote three wrappers that allow slightly nicer handling of tags.

(defun roam-extra:get-filetags ()
  (split-string (or (org-roam-get-keyword "filetags") "")))

(defun roam-extra:add-filetag (tag)
  (let* ((new-tags (cons tag (roam-extra:get-filetags)))
         (new-tags-str (combine-and-quote-strings new-tags)))
    (org-roam-set-keyword "filetags" new-tags-str)))

(defun roam-extra:del-filetag (tag)
  (let* ((new-tags (seq-difference (roam-extra:get-filetags) `(,tag)))
         (new-tags-str (combine-and-quote-strings new-tags)))
    (org-roam-set-keyword "filetags" new-tags-str)))

The layer

roam-extra:todo-p needed no changes at all. I'm including it here only for easy reference.

(defun roam-extra:todo-p ()
  "Return non-nil if current buffer has any TODO entry.

TODO entries marked as done are ignored, meaning the this
function returns nil if current buffer contains only completed
      (org-element-parse-buffer 'headline)
    (lambda (h)
      (eq (org-element-property :todo-type h)
    nil 'first-match))

As pretty much all functions I used in the old version of roam-extra:update-todo-tag are gone I took the opportunity to rework it completely. I think it ended up being slightly simpler. I suspect the the use of org-with-point-at 1 ... is unnecessary, but I haven't tested it yet so I'm leaving it in for now.

(defun roam-extra:update-todo-tag ()
  "Update TODO tag in the current buffer."
  (when (and (not (active-minibuffer-window))
    (org-with-point-at 1
      (let* ((tags (roam-extra:get-filetags))
             (is-todo (roam-extra:todo-p)))
        (cond ((and is-todo (not (seq-contains-p tags "todo")))
               (roam-extra:add-filetag "todo"))
              ((and (not is-todo) (seq-contains-p tags "todo"))
               (roam-extra:del-filetag "todo")))))))

In the previous version roam-extra:todo-files was built using an SQL query. That felt a little brittle to me, so despite that my original inspiration contains an updated SQL query I decided to go the route of using the org-roam API instead. The function org-roam-node-list makes it easy to get all nodes and then finding the files is just a matter of using seq-filter and seq-map. Now that headings may be nodes, and that heading-based nodes seem to inherit the top-level tags, a file may appear more than once, hence the call to seq-unique at the end.

Based on what I've seen V2 appears less eager to sync the DB, so to make sure all nodes are up-to-date it's best to start off with forcing a sync.

(defun roam-extra:todo-files ()
  "Return a list of roam files containing todo tag."
  (let ((todo-nodes (seq-filter (lambda (n)
                                  (seq-contains-p (org-roam-node-tags n) "todo"))
    (seq-uniq (seq-map #'org-roam-node-file todo-nodes))))

With that in place it turns out that also roam-extra:update-todo-files worked without any changes. I'm including it here for easy reference as well.

(defun roam-extra:update-todo-files (&rest _)
  "Update the value of `org-agenda-files'."
  (setq org-agenda-files (roam-extra:todo-files)))

Hooking it up

The variable org-roam-file-setup-hook is gone, so the the more general find-file-hook will have to be used instead.

(add-hook 'find-file-hook #'roam-extra:update-todo-tag)
(add-hook 'before-save-hook #'roam-extra:update-todo-tag)
(advice-add 'org-agenda :before #'roam-extra:update-todo-files)
Tags: emacs org-mode org-roam spacemacs
21 Mar 2021

Todo items in org-roam, an update

I got an email from Mr Z with a nice modification to the code in my post on keeping todo items in org-roam.

He already had a bunch of agenda files that he wanted to keep using (I had so few of them that I'd simply converted them to roam files). Here's the solution he shared with me:

(defvar roam-extra-original-org-agenda-files nil
  "Original value of  `org-agenda-files'.")

(defun roam-extra:update-todo-files (&rest _)
  "Update the value of `org-agenda-files'."
  (unless roam-extra-original-org-agenda-files
    (setq roam-extra-original-org-agenda-files org-agenda-files))
  (setq org-agenda-files
        (append roam-extra-original-org-agenda-files

It's a rather nice modification I think. Thanks to Mr Z for agreeing to let me share it here.

Tags: emacs org-mode org-roam
14 Mar 2021

Keeping todo items in org-roam

A while ago I made an attempt to improve my work habits by keeping a document with TODO items. It lasted only for a while, and I've since had the intention to make another attempt. Since then I've started using org-roam and I've managed to create a habit of writing daily journal notes using org-roam's daily-notes. A few times I've thought that it might fit me well to put TODO items in the notes, but that would mean that I'd have to somehow keep track of them. At first I manually added a tag to each journal fily containing a TODO item. That didn't work very well at all, which should have been obvious up front. Then I added the folders where I keep roam files and journals to org-agenda-files, that worked a lot better. I'd still be using that, even if I expected it to slow down considerably as the number of files grow, but then I found a post on dynamic and fast agenda with org-roam.

I adjusted it slightly to fit my own setup a bit better, i.e. I made a Spacemacs layer, roam-extra, I use the tag todo, and I use a different hook to get the tag added on opening an org-roam file.

The layer consists of a single file, layers/roam-extra/funcs.el. In it I define 4 functions (they are pretty much copies of the functions in the post linked above):

  1. roam-extra:todo-p - returns non-nil if the current current buffer contains a TODO item.
  2. roam-extra:update-todo-tag - updates the tags of the current buffer to reflect the presence of TODO items, i.e. ensure the the tag todo is present iff there's a TODO item.
  3. roam-extra:todo-files - uses the org-roam DB to return a list of all files containing the tag todo.
  4. roam-extra:update-todo-files - adjusts 'org-agenda-files to contain only the files with TODO items.

I've put the full contents of the file at the end of the post.

To ensure that the todo tag is correct in all org-mode files I've added roam-extra:update-todo-tag to hooks that are invoked on opening an org-ram file and when saving a file. (I would love to find a more specialise hook than before-save-hook, but it works for now.)

(add-hook 'org-roam-file-setup-hook #'roam-extra:update-todo-tag)
(add-hook 'before-save-hook #'roam-extra:update-todo-tag)

To ensure that the list of files with TODO items is kept up to date when I open I also wrap org-agenda in an advice so roam-extra:update-todo-files is called prior to the agenda being opened.

(advice-add 'org-agenda :before #'roam-extra:update-todo-files)

The layer, layers/roam-extra/funcs.el

(defun roam-extra:todo-p ()
  "Return non-nil if current buffer has any TODO entry.

TODO entries marked as done are ignored, meaning the this
function returns nil if current buffer contains only completed
      (org-element-parse-buffer 'headline)
    (lambda (h)
      (eq (org-element-property :todo-type h)
    nil 'first-match))

(defun roam-extra:update-todo-tag ()
  "Update TODO tag in the current buffer."
  (when (and (not (active-minibuffer-window))
             (org-roam--org-file-p buffer-file-name))
    (let* ((file (buffer-file-name (buffer-base-buffer)))
           (all-tags (org-roam--extract-tags file))
           (prop-tags (org-roam--extract-tags-prop file))
           (tags prop-tags))
      (if (roam-extra:todo-p)
          (setq tags (seq-uniq (cons "todo" tags)))
        (setq tags (remove "todo" tags)))
      (unless (equal prop-tags tags)
         (combine-and-quote-strings tags))))))

(defun roam-extra:todo-files ()
  "Return a list of note files containing todo tag."
    [:select file
             :from tags
             :where (like tags (quote "%\"todo\"%"))])))

(defun roam-extra:update-todo-files (&rest _)
  "Update the value of `org-agenda-files'."
  (setq org-agenda-files (roam-extra:todo-files)))
Tags: emacs org-mode org-roam spacemacs
05 Mar 2021

Flycheck and HLS

I've been using LSP for most programming languages for a while now. HLS is really very good now, but I've found that it doesn't warn on quite all things I'd like it to so I find myself having to swap between the 'lsp and 'haskell-ghc checkers. However, since flycheck supports chaining checkers I thought there must be a way to have both checkers active at the same time.

The naive approach didn't work due to load order of things in Spacemacs so I had to experiment a bit to find something that works.

The first issue was to make sure that HLS is available at all. I use shell.nix together with direnv extensively and I had noticed that lsp-mode tried to load HLS before direnv had put it in the $PATH. I think the 'lsp-beforeinitialize-hook is the hook to use for this:

(add-hook 'lsp-before-initialize-hook #'direnv-update-environment))

I made a several attempt to chain the checkers but kept on getting errors due to the 'lsp checker not being defined yet. Another problem I ran into was that the checkers were chained too late, resulting in having to manually run flycheck-buffer on the first file I opened. (Deferred loading is a brilliant thing, but make some things really difficult to debug.) After quite a bit of experimenting and reading the description of various hooks I did find something that works:

(with-eval-after-load 'lsp-mode
  (defun magthe:lsp-next-checker ()
    (flycheck-add-next-checker 'lsp '(warning . haskell-ghc)))
  (add-hook 'lsp-lsp-haskell-after-open-hook

Of course I have no idea if this is the easiest or most elegant solution but it does work for my testcases:

  1. Open a file in a project, SPC p l - choose project - choose a Haskell file.
  2. Open a project, SPC p l followed by C-d, and then open a Haskell file.

Suggestions for improvements are more than welcome, of course.

Tags: emacs haskell flycheck
22 Jun 2020

Better Nix setup for Spacemacs

In an earlier post I documented my setup for getting Spacemacs/Emacs to work with Nix. I've since found a much more elegant solution based on

No more Emacs packages for Nix and no need to defining functions that wrap executables in an invocation of nix-shell.

There's a nice bonus too, with this setup I don't need to run nix-shell, which always drops me at a bash prompt, instead I get a working setup in my shell of choice.

Setting up direnv

The steps for setting up direnv depends a bit on your setup, but luckily I found the official instructions for installing direnv to be very clear and easy to follow. There's not much I can add to that.

Setting up Spacemacs

Since emacs-direnv isn't included by default in Spacemacs I needed to do a bit of setup. I opted to create a layer for it, rather than just drop it in the list dotspacemacs-additional-packages. Yes, a little more complicated, but not difficult and I nurture an intention of submitting the layer for inclusion in Spacemacs itself at some point. I'll see where that goes.

For now, I put the following in the file ~/.emacs.d/private/layers/direnv/packages.el:

(defconst direnv-packages

(defun direnv/init-direnv ()
  (use-package direnv

Setting up the project folders

In each project folder I then add the file .envrc containing a single line:


Then I either run direnv allow from the command line, or run the function direnv-allow after opening the folder in Emacs.

Using it

It's as simple as moving into the folder in a shell – all required envvars are set up on entry and unset on exit.

In Emacs it's just as simple, just open a file in a project and the envvars are set. When switching to a buffer outside the project the envvars are unset.

There is only one little caveat, nix-build doesn't work inside a Nix shell. I found out that running

IN_NIX_SHELL= nix-build

does work though.

Tags: emacs nix spacemacs
07 Dec 2019

Nix setup for Spacemacs

Edit 2020-06-22: I've since found a better setup for this.

When using ghcide and LSP, as I wrote about in my post on Haskell, ghcide, and Spacemacs, I found myself ending up recompiling a little too often. This pushed me to finally start looking at Nix. After a bit of a fight I managed to get ghcide from Nix, which brought me the issue of setting up Spacemacs. Inspired by a gist from Samuel Evans-Powell and a guide to setting up an environment for Reflex by Thales Macedo Garitezi I ended up with the following setup:

(defun dotspacemacs/layers ()
(defun dotspacemacs/user-config ()
  (add-hook 'haskell-mode-hook #'lsp)
  (add-hook 'haskell-mode-hook 'nix-haskell-mode)
  (add-hook 'haskell-mode-hook
            (lambda ()
              (setq-local flycheck-executable-find
                          (lambda (cmd)
                            (nix-executable-find (nix-current-sandbox) cmd)))
              (setq-local flycheck-command-wrapper-function
                          (lambda (argv)
                            (apply 'nix-shell-command (nix-current-sandbox) argv)))
              (setq-local haskell-process-wrapper-function
                          (lambda (argv)
                            (apply 'nix-shell-command (nix-current-sandbox) argv)))
              (setq-local lsp-haskell-process-wrapper-function
                          (lambda (argv)
                            `("nix-shell" "-I" "." "--command" "ghcide --lsp" ,(nix-current-sandbox))))))
  (add-hook 'haskell-mode-hook
            (lambda ()
              (flycheck-add-next-checker 'lsp-ui '(warning . haskell-stack-ghc))))

It seems to work, but please let me know if you have suggestions for improvements.

Tags: emacs nix spacemacs
05 Nov 2019

Populating Projectile's cache

As I track the develop branch of Spacemacs I occasionally clean out my cache of projects known to Projectile. Every time it takes a while before I'm back at a stage where I very rarely have to visit something that isn't already in the cache.

However, today I found the function projectile-add-known-project, which prompted me to write the following function that'll help me quickly re-building the cache the next time I need to reset Spacemacs to a known state again.

(defun projectile-extra-add-projects-in-subfolders (projects-root)
  (interactive (list (read-directory-name "Add to known projects: ")))
  (message "Searching for projects in %s..." projects-root)
  (let ((dirs (seq-map 'file-name-directory (directory-files-recursively projects-root "^.git$" t))))
    (seq-do 'projectile-add-known-project dirs)
    (message "Added %d projects" (length dirs))))
Tags: emacs elisp projectile
20 Oct 2019

Ditaa in Org mode

Just found out that Emacs ships with Babel support for ditaa (yes, I'm late to the party).

Sweet! That is yet another argument for converting all our into at work.


The changes I made to my Spacemacs config are

(defun dotspacemacs/user-config ()
  (with-eval-after-load 'org
    (add-to-list 'org-babel-load-languages '(ditaa . t))
    (setq org-ditaa-jar-path "/usr/share/java/ditaa/ditaa-0.11.jar"))
Tags: emacs org-mode
19 Sep 2019

Haskell, ghcide, and Spacemacs

The other day I read Chris Penner's post on Haskell IDE Support and thought I'd make an attempt to use it with Spacemacs.

After running stack build hie-bios ghcide haskell-lsp --copy-compiler-tool I had a look at the instructions on using haskell-ide-engine with Spacemacs. After a bit of trial and error I came up with these changes to my ~/.spacemacs:

(defun dotspacemacs/layers ()
    (haskell :variables
             haskell-completion-backend 'lsp
(defun dotspacemacs/user-config ()
  (setq lsp-haskell-process-args-hie '("exec" "ghcide" "--" "--lsp")
        lsp-haskell-process-path-hie "stack"
        lsp-haskell-process-wrapper-function (lambda (argv) (cons (car argv) (cddr argv)))
  (add-hook 'haskell-mode-hook

The slightly weird looking lsp-haskell-process-wrapper-function is removing the pesky --lsp inserted by this line.

That seems to work. Though I have to say I'm not ready to switch from intero just yet. Two things in particular didn't work with =ghcide=/LSP:

  1. Switching from one the Main.hs in one executable to the Main.hs of another executable in the same project didn't work as expected – I had hints and types in the first, but nothing in the second.
  2. Jump to the definition of a function defined in the package didn't work – I'm not willing to use GNU GLOBAL or some other source tagging system.
Tags: emacs haskell lsp spacemacs
07 May 2019

Some OrgMode stuff

The last few days I've watched Rainer König's OrgMode videos. It's resulted in a few new settings that makes Org a little more useful.

Variable Value Description
calendar-week-start-day 1 Weeks start on Monday!
org-modules (list) org-habit Support for tracking habits
org-modules (list) org-id Improved support for ID property
org-agenda-start-on-weekday 1 Weeks start on Monday, again!
org-log-into-drawer t Put notes (logs) into a drawer
org-enforce-todo-checkbox-dependencies t Checkboxes must be checked before a TODO can become DONE
org-id-link-to-org-use-id t Prefer use of ID property for links
Tags: emacs org-mode
16 Mar 2019

TIL: prompt matters to org-mode

A workmate just embellished some shell code blocks I'd put in a shared org-mode file with :session s. When I tried to run the blocks with sessions my emacs just froze up though. I found a post on the emacs StackExchange that offered a possible cause for it: the prompt.

I'm using bash-it so my prompt is rather far from the default.

After inspecting the session buffer simply added the following to my ~/.bashrc

if [[ ${TERM} == "dumb" ]]; then
    export BASH_IT_THEME='standard'
    export BASH_IT_THEME='simple'

and now I can finally run shell code blocks in sessions.

Tags: emacs org-mode
28 Jan 2019

A missing piece in my Emacs/Spacemacs setup for Haskell development

With the help of a work mate I've finally found this gem that's been missing from my Spacemacs setup

(with-eval-after-load 'intero
  (flycheck-add-next-checker 'intero '(warning . haskell-hlint))
  (flycheck-add-next-checker 'intero '(warning . haskell-stack-ghc)))
Tags: haskell emacs spacemacs
14 Jul 2018

QuickCheck on a REST API

Since I'm working with web stuff nowadays I thought I'd play a little with translating my old post on using QuickCheck to test C APIs to the web.

The goal and how to reach it

I want to use QuickCheck to test a REST API, just like in the case of the C API the idea is to

  1. generate a sequence of API calls (a program), then
  2. run the sequence against a model, as well as
  3. run the sequence against the web service, and finally
  4. compare the resulting model against reality.


I'll use a small web service I'm working on, and then concentrate on only a small part of the API to begin with.

The parts of the API I'll use for the programs at this stage are

Method Route Example in Example out
POST /users {"userId": 0, "userName": "Yogi Berra"} {"userId": 42, "userName": "Yogi Berra"}
DELETE /users/:id    

The following API calls will also be used, but not in the programs

Method Route Example in Example out
GET /users   [0,3,7]
GET /users/:id   {"userId": 42, "userName": "Yogi Berra"}
POST /reset    

Representing API calls

Given the information about the API above it seems the following is enough to represent the two calls of interest together with a constructor representing the end of a program

data ApiCall = AddUser Text
             | DeleteUser Int
             | EndProgram
             deriving (Show)

and a program is just a sequence of calls, so list of ApiCall will do. However, since I want to generate sequences of calls, i.e. implement Arbitrary, I'll wrap it in a newtype

newtype Program = Prog [ApiCall]

Running against a model (simulation)

First of all I need to decide what model to use. Based on the part of the API I'm using I'll use an ordinary dictionary of Int and Text

type Model = M.Map Int Text

Simulating execution of a program is simulating each call against a model that's updated with each step. I expect the final model to correspond to the state of the real service after the program is run for real. The simulation begins with an empty dictionary.

simulateProgram :: Program -> Model
simulateProgram (Prog cs) = foldl simulateCall M.empty cs

The simulation of the API calls must then be a function taking a model and a call, returning an updated model

simulateCall :: Model -> ApiCall -> Model
simulateCall m (AddUser t) = M.insert k t m
    k = succ $ foldl max 0 (M.keys m)
simulateCall m (DeleteUser k) = M.delete k m
simulateCall m EndProgram = m

Here I have to make a few assumptions. First, I assume the indeces for the users start on 1. Second, that the next index used always is the successor of highest currently used index. We'll see how well this holds up to reality later on.

Running against the web service

Running the program against the actual web service follows the same pattern, but here I'm dealing with the real world, so it's a little more messy, i.e. IO is involved. First the running of a single call

runCall :: Manager -> ApiCall -> IO ()
runCall mgr (AddUser t) = do
  ireq <- parseRequest "POST http://localhost:3000/users"
  let req = ireq { requestBody = RequestBodyLBS (encode $ User 0 t)}
  resp <- httpLbs req mgr
  guard (status201 == responseStatus resp)

runCall mgr (DeleteUser k) = do
  req <- parseRequest $ "DELETE http://localhost:3000/users/" ++ show k
  resp <- httpNoBody req mgr
  guard (status200 == responseStatus resp)

runCall _ EndProgram = return ()

The running of a program is slightly more involved. Of course I have to set up the Manager needed for the HTTP calls, but I also need to

  1. ensure that the web service is in a well-known state before starting, and
  2. extract the state of the web service after running the program, so I can compare it to the model
runProgram :: Program -> IO Model
runProgram (Prog cs) = do
  mgr <- newManager defaultManagerSettings
  resetReq <- parseRequest "POST http://localhost:3000/reset"
  httpNoBody resetReq mgr
  mapM_ (runCall mgr) cs
  model <- extractModel mgr
  return model

The call to POST /reset resets the web service. I would have liked to simply restart the service completely, but I failed in automating it. I think I'll have to take a closer look at the implementation of scotty to find a way.

Extracting the web service state and packaging it in a Model is a matter of calling GET /users and then repeatedly calling GET /users/:id with each id gotten from the first call

extractModel :: Manager -> IO Model
extractModel mgr = do
  req <- parseRequest "http://localhost:3000/users"
  resp <- httpLbs req mgr
  let (Just ids) = decode (responseBody resp) :: Maybe [Int]
  users <- forM ids $ \ id -> do
    req <- parseRequest $ "http://localhost:3000/users/" ++ show id
    resp <- httpLbs req mgr
    let (Just (user:_)) = decode (responseBody resp) :: Maybe [User]
    return user
  return $ foldl (\ map (User id name) -> M.insert id name map) M.empty users

Generating programs

My approach to generating a program is based on the idea that given a certain state there is only a limited number of possible calls that make sense. Given a model m it makes sense to make one of the following calls:

  • add a new user
  • delete an existing user
  • end the program

Based on this writing genProgram is rather straight forward

genProgram :: Gen Program
genProgram = Prog <$> go M.empty
    possibleAddUser _ = [AddUser <$> arbitrary]
    possibleDeleteUser m = map (return . DeleteUser) (M.keys m)
    possibleEndProgram _ = [return EndProgram]

    go m = do
      let possibles = possibleDeleteUser m ++ possibleAddUser m ++ possibleEndProgram m
      s <- oneof possibles
      let m' = simulateCall m s
      case s of
        EndProgram -> return []
        _ -> (s:) <$> go m'

Armed with that the Arbitrary instance for Program can be implemented as1

instance Arbitrary Program where
  arbitrary = genProgram
  shrink p = []

The property of an API

The steps in the first section can be used as a recipe for writing the property

prop_progCorrectness :: Program -> Property
prop_progCorrectness program = monadicIO $ do
  let simulatedModel = simulateProgram program
  runModel <- run $ runProgram program
  assert $ simulatedModel == runModel

What next?

There are some improvements that I'd like to make:

  • Make the generation of Program better in the sense that the programs become longer. I think this is important as I start tackling larger APIs.
  • Write an implementation of shrink for Program. With longer programs it's of course more important to actually implement shrink.

I'd love to hear if others are using QuickCheck to test REST APIs in some way, if anyone has suggestions for improvements, and of course ideas for how to implement shrink in a nice way.



Yes, I completely skip the issue of shrinking programs at this point. This is OK at this point though, because the generated =Programs=s do end up to be very short indeed.

Tags: emacs haskell flycheck
Other posts