03 Feb 2024

Bending Warp

In the past I've noticed that Warp both writes to stdout at times and produces some default HTTP responses, but I've never bothered taking the time to look up what possibilities it offers to changes this behaviour. I've also always thought that I ought to find out how Warp handles signals.

If you wonder why this would be interesting to know there are three main points:

  1. The environments where the services run are set up to handle structured logging. In our case it should be JSONL written to stdout, i.e. one JSON object per line.
  2. We've decided that the error responses we produce in our code should be JSON, so it's irritating to have to document some special cases where this isn't true just because Warp has a few default error responses.
  3. Signal handling is, IMHO, a very important part of writing a service that runs well in k8s as it uses signals to handle the lifetime of pods.

Looking through the Warp API

Browsing through the API documentation for Warp it wasn't too difficult to find the interesting pieces, and that Warp follows a fairly common pattern in Haskell libraries

  • There's a function called runSettings that takes an argument of type Settings.
  • The default settings are available in a variable called defaultSettings (not very surprising).
  • There are several functions for modifying the settings and they all have the same shape

    setX :: X -> Settings -> Settings.
    

    which makes it easy to chain them together.

  • The functions I'm interested in now are
    setOnException
    the default handler, defaultOnException, prints the exception to stdout using its Show instance
    setOnExceptionResponse
    the default responses are produced by defaultOnExceptionResponse and contain plain text response bodies
    setInstallShutdownHandler
    the default behaviour is to wait for all ongoing requests and then shut done
    setGracefulShutdownTimeout
    sets the number of seconds to wait for ongoing requests to finnish, the default is to wait indefinitely

Some experimenting

In order to experiment with these I put together a small API using servant, app, with a main function using runSettings and stringing together a bunch of modifications to defaultSettings.

main :: IO ()
main = Log.withLogger $ \logger -> do
    Log.infoIO logger "starting the server"
    runSettings (mySettings logger defaultSettings) (app logger)
    Log.infoIO logger "stopped the server"
  where
    mySettings logger = myShutdownHandler logger . myOnException logger . myOnExceptionResponse

myOnException logs JSON objects (using the logging I've written about before, here and here). It decides wether to log or not using defaultShouldDisplayException, something I copied from defaultOnException.

myOnException :: Log.Logger -> Settings -> Settings
myOnException logger = setOnException handler
  where
    handler mr e = when (defaultShouldDisplayException e) $ case mr of
        Nothing -> Log.warnIO logger $ lm $ "exception: " <> T.pack (show e)
        Just _ -> do
            Log.warnIO logger $ lm $ "exception with request: " <> T.pack (show e)

myExceptionResponse responds with JSON objects. It's simpler than defaultOnExceptionResponse, but it suffices for my learning.

myOnExceptionResponse :: Settings -> Settings
myOnExceptionResponse = setOnExceptionResponse handler
  where
    handler _ =
        responseLBS
            H.internalServerError500
            [(H.hContentType, "application/json; charset=utf-8")]
            (encode $ object ["error" .= ("Something went wrong" :: String)])

Finally, myShutdownHandler installs a handler for SIGTERM that logs and then shuts down.

myShutdownHandler :: Log.Logger -> Settings -> Settings
myShutdownHandler logger = setInstallShutdownHandler shutdownHandler
  where
    shutdownAction = Log.infoIO logger "closing down"
    shutdownHandler closeSocket = void $ installHandler sigTERM (Catch $ shutdownAction >> closeSocket) Nothing

Conclusion

I really ought to have looked into this sooner, especially as it turns out that Warp offers all the knobs and dials I could wish for to control these aspects of its behaviour. The next step is to take this and put it to use in one of the services at $DAYJOB

Tags: haskell warp
09 Dec 2023

Getting Amazonka S3 to work with localstack

I'm writing this in case someone else is getting strange errors when trying to use amazonka-s3 with localstack. It took me rather too long finding the answer and neither the errors I got from Amazonka nor from localstack were very helpful.

The code I started with for setting up the connection looked like this

main = do
  awsEnv <- AWS.overrideService localEndpoint <$> AWS.newEnv AWS.discover
  -- do S3 stuff
  where
    localEndpoint = AWS.setEndpoint False "localhost" 4566

A few years ago, when I last wrote some Haskell to talk to S3 this was enough1, but now I got some strange errors.

It turns out there are different ways to address buckets and the default, which is used by AWS itself, isn't used by localstack. The documentation of S3AddressingStyle has more details.

So to get it to work I had to change the S3 addressing style as well and ended up with this code instead

main = do
  awsEnv <- AWS.overrideService (s3AddrStyle . localEndpoint) <$> AWS.newEnv AWS.discover
  -- do S3 stuff
  where
    localEndpoint = AWS.setEndpoint False "localhost" 4566
    s3AddrStyle svc = svc {AWS.s3AddressingStyle = AWS.S3AddressingStylePath}

Footnotes:

1

That was before version 2.0 of Amazonka, so it did look slightly different, but overriding the endpoint was all that was needed.

Tags: amazonka aws haskell localstack
19 Nov 2023

Making Emacs without terminal emulator a little more usable

After reading Andrey Listopadov's You don't need a terminal emulator (mentioned at Irreal too) I decided to give up on using Emacs as a terminal for my shell. In my experience Emacs simply isn't a very good terminal to run a shell in anyway. I removed the almost completely unused shell-pop from my configuration and the keybinding with a binding to async-shell-command. I'm keeping terminal-here in my config for the time being though.

I realised projectile didn't have a function for running it in the root of a project, so I wrote one heavily based on project-async-shell-command.

(defun mep-projectile-async-shell-command ()
  "Run `async-shell-command' in the current project's root directory."
  (declare (interactive-only async-shell-command))
  (interactive)
  (let ((default-directory (projectile-project-root)))
    (call-interactively #'async-shell-command)))

I quickly found that the completion offered by Emacs for shell-command and async-shell-command is far from as sophisticated as what I'm used to from Z shell. After a bit of searching I found emacs-bash-completion. Bash isn't my shell of choice, partly because I've found the completion to not be as good as in Z shell, but it's an improvement over what stock Emacs offers. The instructions in the repo was good, but had to be adjusted slightly:

(use-package bash-completion
  :straight (:host github
             :repo "szermatt/emacs-bash-completion")
  :config
  (add-hook 'shell-dynamic-complete-functions 'bash-completion-dynamic-complete))

I just wish I'll find a package offering completions reaching Z shell levels.

Tags: emacs
16 Nov 2023

Using the golang mode shipped with Emacs

A few weeks ago I wanted to try out tree-sitter and switched a few of the modes I use for coding to their -ts-mode variants. Based on the excellent How to Get Started with Tree-Sitter I added bits like this to the setup I have for coding modes:1

(use-package X-mode
  :init
  (add-to-list 'treesit-language-source-alist '(X "https://github.com/tree-sitter/tree-sitter-X"))
  ;; (treesit-install-language-grammar 'X)
  (add-to-list 'major-mode-remap-alist '(X-mode . X-ts-mode))
  ;; ...
  )

I then manually evaluated the expression that's commented out to download and compile the tree-sitter grammar. It's a rather small change, it works, and I can switch over language by language. I swapped a couple of languages to the tree-sitter modes like this, including golang. The only mode that I noticed changes in was golang, in particular my adding of gofmt-before-save to before-save-hook had stopped having any effect.

What I hadn't realised was that the go-mode I was using didn't ship with Emacs and that when I switched to go-ts-mode I switched to one that was. It turns out that gofmt-before-save is hard-wired to work only in go-mode, something others have noticed.

I don't feel like waiting for go-mode to fix that though, especially not when there's a perfectly fine golang mode shipping with Emacs now, and not when emacs-reformatter make it so easy to define formatters (as I've written about before).

My golang setup, sans keybindings, now looks like this:2

(use-package go-ts-mode
  :hook
  (go-ts-mode . lsp-deferred)
  (go-ts-mode . go-format-on-save-mode)
  :init
  (add-to-list 'treesit-language-source-alist '(go "https://github.com/tree-sitter/tree-sitter-go"))
  (add-to-list 'treesit-language-source-alist '(gomod "https://github.com/camdencheek/tree-sitter-go-mod"))
  ;; (dolist (lang '(go gomod)) (treesit-install-language-grammar lang))
  (add-to-list 'auto-mode-alist '("\\.go\\'" . go-ts-mode))
  (add-to-list 'auto-mode-alist '("/go\\.mod\\'" . go-mod-ts-mode))
  :config
  (reformatter-define go-format
    :program "goimports"
    :args '("/dev/stdin"))
  :general
  ;; ...
  )

So far I'm happy with the built-in go-ts-mode and I've got to say that using a minor mode for the format-on-save functionality is more elegant than adding a function to before-save-hook (something that go-mode may get through this PR).

Footnotes:

1

There were a few more things that I needed to modify. As the tree-sitter modes are completely separate from the non-tree-sitter modes things like hooks and keybindings in the modes' keymaps.

2

The full file is here.

Tags: emacs tree-sitter
01 Oct 2023

How I use Emacs

I've recently written two posts about my attempts to use a slimmed down Emacs setup for some very specific use cases. I'be put both posts on Reddit, here and here, and in both cases the majority of comments have been telling me that I should use emacsclient. I know they have good intentions – they want to share insight they've gained and benefit from on a daily basis. However, no matter how many Emacs devotees point out the benefits of emacsclient I'm not about to start using it. This post is an attempt to answer why that is.

Up front I want to clarify a few things:

  1. Yes, I know how to use emacsclient, and
  2. yes, I know it is a good way to, in a way, improve Emacs startup time,1 and
  3. yes, I know how to turn on server-mode in Emacs, and finally
  4. yes, I know how to run Emacs using a user unit for SystemD.

With that out of the way, here are the two ways I use Emacs

  1. As my starting point for work, i.e. writing code, keeping notes, tracking time, and writing my daily work journal.
  2. As my editor of ephemeral files.2

The next two sections explain more about these two distinct ways I use Emacs

As my starting point for work

Number of packages 162
Init time (emacs-init-time) 1.883483s
Config size (by du -bch) 68K

Much of what I do on a daily basis starts in Emacs. I typically have one instance of Emacs open and I always keep it on the second virtual desktop. I always run it in the GUI. When I write code I start with opening a new tab, then I open a dired buffer in the project's folder (by using consult-projectile). When I need a terminal I open it from Emacs using on of terminal-here-project-launch or terminal-here. Occasionally I open a shell prompt inside Emacs using shell-pop.

Back when I used Vim as my main editor I always started a terminal first and then opened files from there. Since switching to Emacs I've completely stopped doing that. Over the last 8 or so years of Emacs usage there's only been a handful of times when I've wanted to open a file from the terminal and I've run M-x server-start and used emacsclient. The last time was more than a year ago.

As I typically keep exactly one Emacs open, and I start it soon after logging in, I'm not too concerned with startup time. I think under 2s is more than fast enough given the functionality I have in my setup.

This setup I use for taking notes and writing my daily work journal, as well as reading email, and programming in a half-dozen languages. I have a large-ish set of keybindings that I've set up using general.el, inspired by Spacemacs at first but by now it's started to gain its own character.

As my editor of ephemeral files ($EDITOR)

Number of packages 22
Init time (emacs-init-time) 0.209298s
Config size (by du -bch) 6.7K

Ephemeral files are files I tend to edit for less than 30 seconds, maybe a minute at most. There are three main use cases for ephemeral files:

  1. Searching the scrollback buffer in zellij, and copying bits to the clipboard for various uses.
  2. Editing files when running git from the command line. It's not something I do very often, but it happens.
  3. Editing shell commands. When they get a little too large to handle conveniently using ZSH directly I invoke edit-command-line.

For a few reasons I decided to make a second, completely separate configuration just to handle ephemeral files.

  • It will only be used in a terminal.
  • I want to be able to have some special keybindings that suit a specific use, e.g. for the scrollback buffer I've bound `SPC Y` to copy the selected text to the clipboard and then exit Emacs. It's a thing that I use all the time with the scrollback buffer, but never otherwise.
  • I have no need, nor any desire, to switch from editing a commit message, or searching a scrollback buffer, to reading email or editing an org-mode file. The complete separation is a feature.

With a startup time of less than a quarter of a second it is well within the acceptable, and there is absolutely no need to use emacsclient just to speed things up. Given my desire for separation, I wouldn't want to use my main Emacs instance as a server and edit ephemeral files anyway.

Conclusion

I've found a setup that seems to work really well and tick all the requirements I have when it comes to separation between use cases and ability to have custom keybindings for them. Also, Emacs is starting up very fast with my slimmed down configuration. If starting Emacs with the slimmed configuration starts taking too long I'm more likely to go back to using Neovim than complicate things with emacsclient.

So no, I am not going to start using emacsclient any time soon.

Footnotes:

1

I write "in a way" as it actually does nothing for Emacs startup time, it just shifts it to a point in time so you don't have to sit and wait for it to start.

2

I used to use Neovim, without any config, for most of this until recently.

Tags: emacs
Other posts