Projectile allows using a file, .projectile, in the root of a project. This makes it possible to turn a folder into a project without having to use version control. It's possible to configure project.el to respect more VC markers than what's built-in. This can be used to define a non-VC marker.

(setopt project-vc-extra-root-markers '(".projectile" ".git"))

Since I've set vc-handled-backends to nil (the default made VC interfere with magit, so I turned it off completely) I had to add ".git" to make git repos be recognised as projects too.

The first thing to solve was that the xref stack wasn't per project. Somewhat disappointingly there only seems to be two options for xref-history-storage shipped with Emacs

xref-global-history

a single global history (the default)

xref-window-local-history

a history per window

I had the same issue with projectile, and ended up writing my own package for it. For project.el I settled on using xref-project-history.

(use-package xref-project-history
  :ensure (:type git
           :repo "https://codeberg.org/imarko/xref-project-history.git"
           :branch "master")
  :custom
  (xref-history-storage #'xref-project-history))

Projectile has a function for jumping between implementation and test. Not too surprisingly it's called projectile-toggle-between-implementation-and-test. I found some old emails in an archive suggesting that project.el might have had something similar in the past, but if that's the case it's been removed by now. When searching for a package I came across this email comparing tools for finding related files. The author mentions two that are included with Emacs

ff-find-other-file

part of find-file.el, which a few other functions and a rather impressive set of settings to customise its behaviour.

find-sibling-file

a newer command, I believe, that also can be customised.

So, there are options, but neither of them are made to work nicely with project.el out of the box. My most complicated use case seems to be in Haskell projects where modules for implementation and test live in separate (mirrored) folder hierarchies, e.g.

src
└── Sider
    └── Data
        ├── Command.hs
        ├── Pipeline.hs
        └── Resp.hs
test
└── Sider
    └── Data
        ├── CommandSpec.hs
        ├── PipelineSpec.hs
        └── RespSpec.hs

I'm not really sure how I'd configure find-sibling-rules, which are regular expressions, to deal with folder hierarchies like this. To be honest, I didn't really see a way of configuring ff-find-other-file at first either. Then I happened on a post about switching between a module and its tests in Python. With its help I came up with the following

(defun mes/setup-hs-ff ()
  (when-let* ((proj-root (project-root (project-current)))
              (rel-proj-root (-some--> (buffer-file-name)
                               (file-name-directory it)
                               (f-relative proj-root it)))
              (sub-tree (car (f-split (f-relative (buffer-file-name) proj-root))))
              (search-dirs (--> '("src" "test")
                                (remove sub-tree it)
                                (-map (lambda (p) (f-join proj-root p)) it)
                                (-select #'f-directory? it)
                                (-mapcat (lambda (p) (f-directories p nil t)) it)
                                (-map (lambda (p) (f-relative p proj-root)) it)
                                (-map (lambda (p) (f-join rel-proj-root p)) it))))
    (setq-local ff-search-directories search-dirs
                ff-other-file-alist '(("Spec\\.hs$" (".hs"))
                                      ("\\.hs$" ("Spec.hs"))))))

A few things to note

1. The order of rules in ff-other-file-alist is important, the first match is chosen.
2. (buffer-file-name) can, and really does, return nil at times, and file-name-directory doesn't deal with anything but strings.
3. The entries in ff-search-directories have to be relative to the file in the current buffer, hence the rather involved varlist in the when-let* expression.

With this in place I get the following values for ff-search-directories

src/Sider/Data/Command.hs

("../../../test/Sider" "../../../test/Sider/Data")

test/Sider/Data/CommandSpec.hs

("../../../src/Sider" "../../../src/Sider/Data")

And ff-find-other-file works beautifully.

My setup with project.el now covers everything I used from projectile so I'm fairly confident I'll be happy keeping it.

Since eglot it's written to work with other packages in core, which means it integrates with flymake. The switch comprised

• Use :ensure nil to make sure elpaca knows there's nothing to download.
• Add a call to flymake-mode to prog-mode-hook.
• Define two functions to toggle showing a list of diagnostics for the current buffer and the project.
• Redefine the relevant keybindings.

The two functions for toggling showing diagnostics look like this

(defun mes/toggle-flymake-buffer-diagnostics ()
  (interactive)
  (if-let* ((window (get-buffer-window (flymake--diagnostics-buffer-name))))
      (save-selected-window (quit-window nil window))
    (flymake-show-buffer-diagnostics)))

(defun mes/toggle-flymake-project-diagnostics ()
  (interactive)
  (if-let* ((window (get-buffer-window (flymake--project-diagnostics-buffer (projectile-project-root)))))
      (save-selected-window (quit-window nil window))
    (flymake-show-project-diagnostics)))

And the changed keybindings are

When it comes to use-package I keep on being surprised, and after the switch to elpaca I've found some new surprises. One of them was that using :after eglot like this

(use-package haskell-ng-mode
  :afer eglot
  :ensure (:type git
           :repo "git@gitlab.com:magus/haskell-ng-mode.git"
           :branch "main")
  :init
  (add-to-list 'major-mode-remap-alist '(haskell-mode . haskell-ng-mode))
  (add-to-list 'eglot-server-programs '(haskell-ng-mode "haskell-language-server-wrapper" "--lsp"))
  (setq-default eglot-workspace-configuration
                (plist-put eglot-workspace-configuration
                           :haskell
                           '(:formattingProvider "fourmolu"
                             :plugin (:stan (:global-on :json-false)))))
  ...
  :hook
  (haskell-ng-mode . eglot-ensure)
  ...)

would delay initialisation until after eglot had been loaded. However, it turned out that nothing in :init ... seemed to run and upon opening a haskell file no mode was loaded.

After a bit of thinking and tinkering I got it working by removing :after eglot and using with-eval-after-load

(use-package haskell-ng-mode
  :ensure (:type git
           :repo "git@gitlab.com:magus/haskell-ng-mode.git"
           :branch "main")
  :init
  (add-to-list 'major-mode-remap-alist '(haskell-mode . haskell-ng-mode))
  (with-eval-after-load 'eglot
    (add-to-list 'eglot-server-programs '(haskell-ng-mode "haskell-language-server-wrapper" "--lsp"))
    (setq-default eglot-workspace-configuration
                  (plist-put eglot-workspace-configuration
                             :haskell
                             '(:formattingProvider "fourmolu"
                               :plugin (:stan (:global-on :json-false))))))
  ...
  :hook
  (haskell-ng-mode . eglot-ensure)
  ...)

That change worked for haskell, and it seemed to work for python too, but after a little while I realised that python needed a bit more attention.

The python setup looked like this

(use-package python
  :init
  (add-to-list 'major-mode-remap-alist '(python-mode . python-ts-mode))
  (with-eval-after-load 'eglot
    (assoc-delete-all '(python-mode python-ts-mode) eglot-server-programs)
    (add-to-list 'eglot-server-programs
                 `((python-mode python-ts-mode) . ,(eglot-alternatives
                                                    '(("rass" "python") "pylsp")))))
  ...
  :hook (python-ts-mode . eglot-ensure)
  ...)

and it worked all right, but then I visited the package (using elpaca-visit) and realised that the downloaded package was all of emacs. That's a bit of overkill, I'd say.

However, adding :ensure nil didn't have the expected effect of just using the version that's in core. Instead the whole configuration seemed to never take effect and again I was back to the situation where I had to jump to python-ts-mode manually.

The documentation for use-package says that :init is for

Code to run before PACKAGE-NAME has been loaded.

but I'm guessing "before" isn't quite before enough. Then I noticed :preface with the description

Code to be run before everything except :disabled; this can be used to define functions for use in :if, or that should be seen by the byte-compiler.

and yes, "before everything" is early enough. The final python configuration looks like this

(use-package python
  :ensure nil
  :preface
  (add-to-list 'major-mode-remap-alist '(python-mode . python-ts-mode))
  :init
  (with-eval-after-load 'eglot
    (assoc-delete-all '(python-mode python-ts-mode) eglot-server-programs)
    (add-to-list 'eglot-server-programs
                 `((python-mode python-ts-mode) . ,(eglot-alternatives
                                                    '(("rass" "python") "pylsp")))))
  ...
  :hook (python-ts-mode . eglot-ensure)
  ...)

I'm still not sure I have the correct intuition about how to use use-package, but hopefully it's more correct now than before. I have a growing suspicion that use-package changes behaviour based on the package manager I use. Or maybe it's just that some package managers make use-package more forgiving of bad use.

I removed the statements pulling in lsp-mode, lsp-ui, and all language-specific packages like lsp-haskell. Then I added this to configure eglot

(use-package eglot
  :ensure nil
  :custom
  (eglot-autoshutdown t)
  (eglot-confirm-server-edits '((eglot-rename . nil)
                                (t . diff))))

The rest was mainly just switching lsp-mode functions for eglot functions.

I haven't verified that the list is fully correct yet, but it looks good so far.

The one thing I might miss is lenses, and using lsp-avy-lens. However, everything that I use lenses for can be done using actions, and to be honest I don't think I'll miss the huge lens texts from missing type annotations in Haskell.

One good thing about lsp-mode's use of language-specific packages is that configuration of the various servers is performed through functions. This makes it easy to discover what options are available, though it also means not all options may be available. In eglot configuration is less organised, I have to know about the options for each language server and put the options into eglot-workspace-configuration myself. It's not always easy to track down what options are available, and I've found no easy way to verify the settings. For instance, with lsp-mode I configures HLS like this

(lsp-haskell-formatting-provider "fourmolu")
(lsp-haskell-plugin-stan-global-on nil)

which translates to this for eglot

(setq-default eglot-workspace-configuration
              (plist-put eglot-workspace-configuration
                         :haskell
                         '(:formattingProvider "fourmolu"
                           :plugin (:stan (:global-on :json-false)))))

and I can verify that this configuration has taken effect because I know enough about the Haskell tools.

I do some development in Python and I used to configure pylsp like this

(lsp-pylsp-plugins-mypy-enabled t)
(lsp-pylsp-plugins-ruff-enabled t)

which I think translates to this for eglot

(setq-default eglot-workspace-configuration
              (plist-put eglot-workspace-configuration
                         :pylsp
                         '(:plugins (:ruff (:enabled t)
                                     :mypy (:enabled t)))))

but I don't know any convenient way of verifying these settings. I'm simply not familiar enough with the Python tools. I can check the value of eglot-workspace-configuration by inspecting it or calling eglot-show-workspace-configuration but is there really no way of asking the language server for its active configuration?

The last time I gave up on eglot very quickly, probably too quickly to be honest. I made these changes to my configuration over the weekend, so the real test of eglot starts when I'm back in the office. I have a feeling I'll stick to it longer this time.

I give the constructor the result type Either String Person to make sure it can both be usable in code and when defining parseJSON.

mkPerson :: Text -> Int -> Occupation -> Either String Person
mkPerson name age occupation = do
    guardE mustBeUnderAge
    guardE notUnderAge
    guardE tooOldToBeStudent
    guardE mustBeRetired
    pure $ Person name age occupation
  where
    guardE (pred, err) = when pred $ Left err
    mustBeUnderAge = (age < 8 && occupation > UnderAge, "too young for occupation")
    notUnderAge = (age > 15 && occupation == UnderAge, "too old to be under age")
    tooOldToBeStudent = (age > 45 && occupation == Student, "too old to be a student")
    mustBeRetired = (age > 65 && occupation /= Retired, "too old to not be retired")

Here I'm making use of Either e being a Monad and use when to apply the constraints and ensure the reason for failure is given to the caller.

When defining the instance I take advantage of the Generic instance to make the implementation short and simple.

instance FromJSON Person where
    parseJSON v = do
        Person{name, age, occupation} <- genericParseJSON defaultOptions v
        either fail pure $ mkPerson name age occupation

If there are many more fields in the type I'd consider using RecordWildCards.

No, it's nothing ground-breaking but I think it's a fairly nice example of how things can fit together in Haskell.

I read about how to create a catppuccin-mocha theme using modus-vivendi through modus' mechanism of overrides. On Reddit someone pointed out that Prot had been working on basing themes on modus and when I checked the state of it he'd just released version 5.0.0. Since I'm using catppuccin themes for pretty much all software with a GUI I thought it could be interesting to see if I could make a modus-based catppuccin theme to replace my use of catppuccin-theme.

I'm writing the rest as if it was a straight and easy journey. It wasn't! I made a few false starts, each time realising something new about the structure and starting over with a better idea.

When reading what Prot had written about modus-themes in general, and about how to create themes based on it, in particular, I found that he's ported both standard-themes and ef-themes so they now are based on modus. Instead of just using them for inspiration I decided that since standard-themes is so small I might as well use it as my starting point.

I copied all files of standard-themes to an empty git repository, then I

• deleted all but one of the theme file
• copied the remaining theme file so I had four in total (one for each of the catppuccin flavours)
• renamed constants, variables, and functions so they would match the theme and its flavours
• put the colours into each catppuccin-<flavour>-palette
• emptied the common palette, modus-catppuccin-common-palette-mappings
• made sure that my use of modus-themes-theme was reasonable, in particular the base palette (I based the light flavour on modus-operandi and the three dark flavours on modus-vivendi)

The result can be seen here.

At this point the three theme flavours contained no relevant mappings of their own, so what I had was in practice modus-operandi under a new name and modus-vivendi under three new names.

By organising the theme flavours the way outlined above I only need to add mappings to modus-catppuccin-common-palette-mappings because

1. each flavour-specific mapping adds its colour palette using the same name (that's how catppuccin organises its colors too, as seen here)
2. each flavour-specific mapping is combined with the common one
3. any missing mapping is picked up by the underlying theme, modus-operandi or modus-vivendi, so there will be (somewhat) nice colours for everything

I started out with the mappings in the dark standard theme but then I realised that's not the complete list of available mappings and I started looking at the themes in modus-themes itself.

I've so far defined enough mappings to make it look enough like catppuccin for my use. There are a lot of possible mappings so my plan is to add them over time and use catppuccin-theme for inspiration.

The Redis protocol has some defining attributes

• It's somewhat of a binary protocol. If you stick to keys and values that fall within the set of ASCII strings, then the protocol is humanly readable and you can rather easily use netcat or telnet as a client. However, you aren't limited to storing only readable strings.
• It's somewhat of a request-response protocol. A notable exception is the publish-subscribe subset, but it's rather small and I reckon most Redis users don't use it.
• It's somewhat of a type-length-value style protocol. Some of the data types include their length in bytes, e.g. bulk strings and verbatim strings. Other types include the number of elements, e.g. arrays and maps. A large number of them have no length at all, e.g. simple strings, integers, and doubles.

I suspect there are good reasons, I gather a lot of it has to do with speed. It does however cause one issue when writing a client: it's not possible to read a whole response without parsing it.

With that extra information about the RESP3 protocol the naïve implementation above falls short in a few ways

• The read buffer may contain more than one full message and give the definition of decode above any remaining bytes are simply dropped.¹
• The read buffer my contain less than one full message and then decode will return an error.²

Surely this must be solvable, because in my mind running the parser results in one of three things:

1. Parsing is done and the result is returned, together with any input that wasn't consumed.
2. The parsing is not done due to lack of input, this is typically encoded as a continuation.
3. The parsing failed so the error is returned, together with input that wasn't consumed.

So, I started looking in the documentation for the module Data.Attoparsec.ByteString.Lazy in attoparsec. I was a little surprised to find that the Result type lacked a way to feed more input to a parser – it only has two constructors, Done and Fail:

data Result r
    = Fail ByteString [String] String
    | Done ByteString r

I'm guessing the idea is that the function producing the lazy bytestring in the first place should be able to produce more chunks of data on demand. That's likely what the lazy variant of recv does, but at the same time it also requires choosing a maximum length and that doesn't rhyme with RESP3. The lazy recv isn't quite lazy in the way I needed it to be.

When looking at the parser for strict bytestrings I calmed down. This parser follows what I've learned about parsers (it's not defined exactly like this; it's parameterised in its input but for the sake of simplicity I show it with ByteString as input):

data Result r
    = Fail ByteString [String] String
    | Partial (ByteString -> Result r)
    | Done ByteString r

Then to my delight I found that there's already a function for handling exactly my problem

parseWith :: Monad m => (m ByteString) -> Parser a -> ByteString -> m (Result a)

I only needed to rewrite the existing parser to work with strict bytestrings and work out how to write a function using recv (for strict bytestrings) that fulfils the requirements to be used as the first argument to parseWith. The first part wasn't very difficult due to the similarity between attoparsec's APIs for lazy and strict bytestrings. The second only had one complication. It turns out recv is blocking, but of course that doesn't work well with parseWith. I wrapped it in timeout based on the idea that timing out means there's no more data and the parser should be given an empty string so it finishes. I also decided to pass the parser as an argument, so I could use the same function for receiving responses for individual commands as well as for pipelines. The full receiving function is

import Data.ByteString qualified as BS
import Data.Text qualified as T
import Network.Socket.ByteString qualified as SB

recvParse :: S.Socket -> Parser r -> IO (Either Text (BS.ByteString, r))
recvParse sock parser = do
    parseWith receive parser BS.empty >>= \case
        Fail _ [] err -> pure $ Left (T.pack err)
        Fail _ ctxs err -> pure $ Left $ T.intercalate " > " (T.pack <$> ctxs) <> ": " <> T.pack err
        Partial _ -> pure $ Left "impossible error"
        Done rem result -> pure $ Right (rem, result)
  where
    receive =
        timeout 100_000 (SB.recv sock 4096) >>= \case
            Nothing -> pure BS.empty
            Just bs -> pure bs

Then I only needed to rewrite sendCmd and I wanted to do it in such a way that any remaining input data could be use in by the next call to sendCmd.³ I settled for modifying the Conn type to hold an IORef ByteString together with the socket and then the function ended up looking like this

sendCmd :: Conn -> Command r -> IO (Result r)
sendCmd (Conn p) (Command k cmd) = withResource p $ \(sock, remRef) -> do
    _ <- SBL.send sock $ toWireCmd cmd
    rem <- readIORef remRef
    recvParse sock rem resp >>= \case
        Left err -> pure $ Left $ RespError "recv/parse" err
        Right (newRem, r) -> do
            writeIORef remRef newRem
            pure $ k <$> fromWireResp cmd r

I've started looking into pub/sub, and basically all of the work described in this post is a prerequisite for that. It's not very difficult on the protocol level, but I think it's difficult to come up with a design that allows maximal flexibility. I'm not even sure it's worthwhile the complexity.

The Redis Serialization Protocol (RESP) initially reminded me of JSON and I thought that following the pattern of aeson might be a good idea. I decided up-front that I'd only support the latest version of RESP, i.e. version 3. So, I thought of a data type, Resp with a constructor for each RESP3 data type, and a pair of type classes, FromResp and ToResp for converting between Haskell types and RESP3. Then after some more reflection I realised that converting to RESP is largely pointless. The main reason to convert anything to RESP3 is to assemble a command, with its arguments, to send to Redis, but all commands are arrays of bulk strings so it's unlikely that anyone will actually use ToResp.¹ So I scrapped the idea of ToResp. FromResp looked like this

class FromResp a where
    fromResp :: Value -> Either FromRespError a

When I started defining commands I didn't like the number of ByteString arguments that resulted in, so I defined a data type, Arg, and an accompanying type class for arguments, ToArg:

newtype Arg = Arg {unArg :: [ByteString]}
    deriving (Show, Semigroup, Monoid)

class ToArg a where
    toArg :: a -> Arg

Later on I saw that it might also be nice to have a type class specifically for keys, ToKey, though that's a wrapper for a single ByteString.

Implementing the functions to encode/decode the protocol were straight-forward applications of attoparsec and bytestring (using its Builder).

Even though supporting pipelining was one of the goals I felt a need to make sure I'd understood the protocol so I started off with single commands. The protocol is a simple request/response protocol at the core so I settled on this type for commands

type Cmd a = forall m. (Monad m) => (ByteString -> m ByteString) -> m (Either FromRespError a)

that is, a command is a function accepting a sender and returning an a.

I wrote a helper function for defining commands, sendCmd

sendCmd :: (Monad m, FromResp a) => [ByteString] -> (ByteString -> m ByteString) -> m (Either FromRespError a)
sendCmd cmdArgs send = do
    let cmd = encode $ Array $ map BulkString cmdArgs
    send cmd <&> decode >>= \case
        Left desc -> pure $ Left $ FromRespError "Decode" (Text.pack desc)
        Right v -> pure $ fromValue v

which made it easy to define commands. Here are two examples, append and mget:

append :: (ToArg a, ToArg b) => a -> b -> Cmd Int
append key val = sendCmd $ ["APPEND"] <> unArg (toArg key <> toArg val)

-- | https://redis.io/docs/latest/commands/mget/
mget :: (ToArg a, FromResp b) => NE.NonEmpty a -> Cmd (NE.NonEmpty b)
mget ks = sendCmd $ ["MGET"] <> unArg (foldMap1 toArg ks)

The function to send off a command and receive its response, sendAndRecieve, was just a call to send followed by a call to recv in network (the variants for lazy bytestrings).

I sort of liked this representation – there's always something pleasant with finding a way to represent something as a function. There's a very big problem with it though: it's difficult to implement pipelining!

Yes, Cmd is a functor since (->) r is a functor, and thus it's possible to make it an Applicative, e.g. using free. However, to implement pipelining it's necessary to

1. encode all commands, then
2. concatenate them all into a single bytestring and send it
3. read the response, which is a concatenation of the individual commands' responses, and
4. convert each separate response from RESP3.

That isn't easy when each command contains its own encoding and decoding. The sender function would have to relinquish control after encoding the command, and resume with the resume again later to decode it. I suspect it's doable using continuations, or monad-coroutine, but it felt complicated and rather than travelling down that road I asked for ideas on the Haskell Discourse. The replies lead me to a paper, Free delivery, and a bit later a package, monad-batcher. When I got the pointer to the package I'd already read the paper and started implementing the ideas in it, so I decided to save exploring monad-batcher for later.

The paper Free delivery is a perfect match for pipelining in Redis, and my understanding is that it proposes a solution where

1. Commands are defined as a GADT, Command a.
2. Two functions are defined to serialise and deserialise a Command a. In the paper they use String as the serialisation, so show and read is used.
3. A type, ActionA a, is defined that combines a command with a modification of its a result. It implements Functor.
4. A free type, FreeA f a is defined, and made into an Applicative with the constraint that f is a Functor.
5. A function, serializeA, is defined that traverses a FreeA ActionA a serialising each command.
6. A function, deserializeA, is defined that traverses a FreeA ActionA a deserialising the response for each command.

I defined a command type, Command a, with only three commands in it, echo, hello, and ping. I then followed the recipe above to verify that I could get it working at all. The Haskell used in the paper is showing its age, and there seems to be a Functor instance missing, but it was still straight forward and I could verify that it worked against a locally running Redis.

Then I made a few changes…

I renamed the command type to Cmd so I could use Command for what the paper calls ActionA.

data Cmd r where
    Echo :: Text -> Cmd Text
    Hello :: Maybe Int -> Cmd ()
    Ping :: Maybe Text -> Cmd Text

data Command a = forall r. Command !(r -> a) !(Cmd r)

instance Functor Command where
    fmap f (Command k c) = Command (f . k) c

toWireCmd :: Cmd r -> ByteString
toWireCmd (Echo msg) = _
toWireCmd (Hello ver) = _
toWireCmd (Ping msg) = _

fromWireResp :: Cmd r -> Resp -> Either RespError r
fromWireResp (Echo _) = fromResp
fromWireResp (Hello _) = fromResp
fromWireResp (Ping _) = fromResp

(At this point I was still using FromResp.)

I also replaced the free applicative defined in the paper and started using free. A couple of type aliases make it a little easier to write nice signatures

type Pipeline a = Ap Command a

type PipelineResult a = Validation [RespError] a

and defining individual pipeline commands turned into something rather mechanical. (I also swapped the order of the arguments to build a Command so I can use point-free style here.)

liftPipe :: (FromResp r) => Cmd r -> Pipeline r
liftPipe = liftAp . Command id

echo :: Text -> Pipeline Text
echo = liftPipe . Echo

hello :: Maybe Int -> Pipeline ()
hello = liftPipe . Hello

ping :: Maybe Text -> Pipeline Text
ping = liftPipe . Ping

One nice thing with switching to free was that serialisation became very simple

toWirePipeline :: Pipeline a -> ByteString
toWirePipeline = runAp_ $ \(Command _ c) -> toWireCmd c

On the other hand deserialisation became a little more involved, but it's not too bad

fromWirePipelineResp :: Pipeline a -> [Resp] -> PipelineResult a
fromWirePipelineResp (Pure a) _ = pure a
fromWirePipelineResp (Ap (Command k c) p) (r : rs) = fromWirePipelineResp p rs <*> (k <$> liftError singleton (fromWireResp c r))
fromWirePipelineResp _ _ = Failure [RespError "fromWirePipelineResp" "Unexpected wire result"]

Everything was working nicely and I started adding support for more commands. I used the small service from work to guide my choice of what commands to add. First out was del, then get and set. After adding lpush I was pretty much ready to try to replace hedis in the service from work.

data Cmd r where
    -- echo, hello, ping
    Del :: (ToKey k) => NonEmpty k -> Cmd Int
    Get :: (ToKey k, FromResp r) => k -> Cmd r
    Set :: (ToKey k, ToArg v) => k -> v -> Cmd Bool
    Lpush :: (ToKey k, ToArg v) => k -> NonEmpty v -> Cmd Int

However, when looking at the above definition started I thinking.

• Was it really a good idea to litter Cmd with constraints like that?
• Would it make sense to keep the Cmd type a bit closer to the actual Redis commands?
• Also, maybe FromResp wasn't such a good idea after all, what if I remove it?

That brought me to the third version of the type for Redis commands.

While adding new commands and writing instances of FromResp I slowly realised that my initial thinking of RESP3 as somewhat similar to JSON didn't really pan out. I had quickly dropped ToResp and now the instances of FromResp didn't sit right with me. They obviously had to "follow the commands", so to speak, but at the same time allow users to bring their own types. For instance, LSPUSH returns the number of pushed messages, but at the same time GET should be able to return an Int too. This led to Int's FromResp looking like this

instance FromResp Int where
    fromResp (BulkString bs) =
        case parseOnly (AC8.signed AC8.decimal) bs of
            Left s -> Left $ RespError "FromResp" (TL.pack s)
            Right n -> Right n
    fromResp (Number n) = Right $ fromEnum n
    fromResp _ = Left $ RespError "FromResp" "Unexpected value"

I could see this becoming worse, take the instance for Bool, I'd have to consider that

• for MOVE Integer 1 means True and Integer 0 means False
• for SET SimpleString "OK" means True
• users would justifiably expect a bunch of bytestrings to be True, e.g. BulkString "true", BulkString "TRUE", BulkString "1", etc

However, it's impossible to cover all ways users can encode a Bool in a ByteString so no matter what I do users will end up having to wrap their Bool with newtype and implement a fitting FromResp. On top of that, even thought I haven't found any example of it yet, I fully expect there to be, somewhere in the large set of Redis commands, at least two commands each wanting an instance of a basic type that simply can't be combined into a single instance, meaning that the client library would need to do some newtype wrapping too.

No, I really didn't like it! So, could I get rid of FromResp and still offer users an API where they can user their own types as the result of commands?

To be concrete I wanted this

data Cmd r where
    -- other commands
    Get :: (ToKey k) => k -> Cmd (Maybe ByteString)

and I wanted the user to be able to conveniently turn a Cmd r into a Cmd s. In other words, I wanted a Functor instance. Making Cmd itself a functor isn't necessary and I just happened to already have a functor type that wraps Cmd, the Command type I used for pipelining. If I were to use that I'd need to write wrapper functions for each command though, but if I did that then I could also remove the ToKey~/~ToArg constraints from the constructors of Cmd r and put them on the wrapper instead. I'd get

data Cmd r where
    -- other commands
    Get :: Key -> Cmd (Maybe ByteString)

get :: (ToKey k) => k -> Command (Maybe ByteString)
get = Command id . Get . toKey

I'd also have to rewrite fromWireResp so it's more specific for each command. Instead of

fromWireResp :: Cmd r -> Resp -> Either RespError r
fromWireResp (Get _) = fromResp
...

I had to match up exactly on the possible replies to GET

fromWireResp :: Cmd r -> Resp -> Either RespError r
fromWireResp _ (SimpleError err desc) = Left $ RespError (T.decodeUtf8 err) (T.decodeUtf8 desc)
fromWireResp (Get _) (BulkString bs) = Right $ Just bs
fromWireResp (Get _) Null = Right Nothing
...
fromWireResp _ _ = Left $ RespError "fromWireResp" "Unexpected value"

Even though it was more code I liked it better than before, and I think it's slightly simpler code. I also hope it makes the use of the API is a bit simpler and clear.

Here's an example from the code for the service I wrote for work. It reads a UTC timestamp stored in timeKey, the timestamp is a JSON string so it needs to be decoded.

readUTCTime :: Connection -> IO (Maybe UTCTime)
readUTCTime conn =
    sendCmd conn (maybe Nothing decode <$> get timeKey) >>= \case
        Left _ -> pure Nothing
        Right datum -> pure datum

I'm pretty happy with the command type for now, though I have a feeling I'll have to revisit Arg and ToArg at some point.

I've just turned the Connection type into a pool using resource-pool, and I started looking at pub/sub. The latter thing, pub/sub, will require some thought and experimentation I think. Quite possibly it'll end up in a post here too.

I also have a lot of commands to add.