Why I'm writing a Redis client package

A couple of weeks ago I needed a small, hopefully temporary, service at work. It bridges a gap in functionality provided by a legacy system and the functionality desired by a new system. The legacy system is cumbersome to work with, so we tend to prefer building anti-corruption layers rather than changing it directly, and sometimes we implement it as separate services.

This time it was good enough to run the service as a cronjob, but it did need to keep track of when it ran the last time. It felt silly to spin up a separate DB just to keep a timestamp, and using another service's DB is something I really dislike and avoid.¹ So, I ended up using the Redis instance that's used as a cache by a OSS service we host.

The last time I had a look at the options for writing a Redis client in Haskell I found two candidates, hedis and redis-io. At the time I wrote a short note about them. This time around I found nothing much has changed, they are still the only two contenders and they still suffer from the same issues

• hedis has still has the same API and I still find it as awkward.
• redis-io still requires a logger.

I once again decided to use hedis and wrote the service for work in a couple of days, but this time I thought I'd see what it would take to remove the requirement on tinylog from redis-io. I spent a few evenings on it, though I spent most time on "modernising" the dev setup, using Nix to build, re-format using fourmolu, etc. I did the same for redis-resp, the main dependency of redis-io. The result of that can be found on my gitlab account:

• https://gitlab.com/magus/redis-resp
• https://gitlab.com/magus/redis-io

At the moment I won't take that particular experiment any further and given that the most recent change to redis-io was in 2020 (according to its git repo) I don't think there's much interest upstream either.

Making the changes to redis-io and redis-resp made me a little curious about the Redis protocol so I started reading about it. It made me start thinking about implementing a client lib myself. How hard could it be?

I'd also asked a question about Redis client libs on r/haskell and a response led me to redis-schema. It has a very good README, and its section on transactions with its observation that Redis transactions are a perfect match for Applicative. This pushed me even closer to start writing a client lib. What pushed me over the edge was the realisation that pipelining also is a perfect match for Applicative.

For the last few weeks I've spent some of my free time reading and experimenting and I'm enjoying it very much. We'll see where it leads, but hopefully I'll at least have bit more to write about it.

Using lens-aeson to implement FromJSON

At work I sometimes need to deal with large and deep JSON objects where I'm only interested in a few of the values. If all the interesting values are on the top level, then aeson have functions that make it easy to implement FromJSON's parseJSON (Constructors and accessors), but if the values are spread out then the functions in aeson come up a bit short. That's when I reach for lens-aeson, as lenses make it very easy to work with large structures. However, I've found that using its lenses to implement parseJSON become a lot easier with a few helper functions.

Many of the lenses produces results wrapped in Maybe, so the first function is one that transforms a Maybe a to a Parser a. Here I make use of Parser implementing MonadFail.

infixl 8 <!>
(<!>) :: (MonadFail m) => Maybe a -> String -> m a
(<!>) mv err = maybe (fail err) pure mv

In some code I wrote this week I used it to extract the user name out of a JWT produced by Keycloak:

instance FromJSON OurClaimsSet where
    parseJSON = ... $ \o -> do
        cs <- parseJSON o
        n <- o ^? key "preferred_username" . _String <!> "preferred username missing"
        ...
        pure $ OurClaimsSet cs n ...

Also, all the lenses start with a Value and that makes the withX functions in aeson to not be a perfect fit. So I define variations of the withX functions, e.g.

withObjectV :: String -> (Value -> Parser a) -> Value -> Parser a
withObjectV s f = withObject s (f . Object)

That makes the full FromJSON instance for OurClaimsSet look like this

instance FromJSON OurClaimsSet where
    parseJSON = withObjectV "OurClaimsSet" $ \o -> do
        cs <- parseJSON o
        n <- o ^? key "preferred_username" . _String <!> "name"
        let rs = o ^.. key "resource_access" . members . key "roles" . _Array . traverse . _String
        pure $ OurClaimsSet cs n rs

Reviewing GitHub PRs in Emacs

My Emacs config's todo-list has long had an item about finding some way to review GitHub PRs without having to leave Emacs and when the forge issue that I subscribe to came alive again I thought it was time to see if I can improve my config.

I found three packages for doing reviews

• code-review
• github-review
• emacs-pr-review

I've tried the first one before but at the time it didn't seem to work at all. Apparently that's improved somewhat, though there's a PR with a change that's necessary to make it work.¹ The first two don't support comments on multiple lines of a PR, there are issues/discussions for both

• code-review: Code suggestion on multiple lines
• github-review: Multi-line code comments

The last one, emacs-pr-review does support commenting on multiple lines, but it lacks a nice way of opening a review from magit. What I can do is

1. position the cursor on a PR in the magit status view, then
2. copy the the PR's URL using forge-copy-url-at-point-as-kill, and
3. open the PR by calling pr-review and pasting the PR's URL.

Which I did for a few days until I got tired of it and wrote a function to cut out they copy/paste part.

(defun mes/pr-review-via-forge ()
  (interactive)
  (if-let* ((target (forge--browse-target))
            (url (if (stringp target) target (forge-get-url target)))
            (rev-url (pr-review-url-parse url)))
      (pr-review url)
    (user-error "No PR to review at point")))

I've bound it to a key in magit-mode-map to make it easier.

I have to say I'm not completely happy with emacs-pr-review, so if either of the other two sort out commenting on multiple lines I'll check them out again.

My full setup for pr-review is here.

Servant and a weirdness in Keycloak

When writing a small tool to interface with Keycloak I found an endpoint that require the content type to be application/json while the body should be plain text. (The details are in the issue.) Since servant assumes that the content type and the content match (I know, I'd always thought that was a safe assumption to make too) it doesn't work with ReqBody '[JSON] Text. Instead I had to create a custom type that's a combination of JSON and PlainText, something that turned out to required surprisingly little code:

data KeycloakJSON deriving (Typeable)

instance Accept KeycloakJSON where
    contentType _ = "application" // "json"

instance MimeRender KeycloakJSON Text where
    mimeRender _ = fromStrict . encodeUtf8

The bug has already been fixed in Keycloak, but I'm sure there are other APIs with similar weirdness so maybe this will be useful to someone else.

Followup on secrets in my work notes

I got the following question on my post on how I handle secrets in my work notes:

Sounds like a nice approach for other secrets but how about :dbconnection for Orgmode and sql-connection-alist?

I have to admit I'd never come across the variable sql-connection-alist before. I've never really used sql-mode for more than editing SQL queries and setting up code blocks for running them was one of the first things I used yasnippet for.

I did a little reading and unfortunately it looks like sql-connection-alist can only handle string values. However, there is a variable sql-password-search-wallet-function, with the default value of sql-auth-source-search-wallet, so using auth-source is already supported for the password itself.

There seems to be a lack of good tutorials for setting up sql-mode in a secure way – all articles I found place the password in clear-text in the config – filling that gap would be a nice way to contribute to the Emacs community. I'm sure it'd prompt me to re-evaluate incorporating sql-mode in my workflow.