Magnus web site
Random stuff
Update to Hackage revisions in Nix
A few days after I published Hackage revisions in Nix I got a comment from
Wolfgang W that the next release of Nix will have a callHackageDirect
with
support for specifying revisions.
The code in PR #284490 makes callHackageDirect
accept a rev
argument. Like
this:
haskellPackages.callHackageDirect { pkg = "openapi3"; ver = "3.2.3"; sha256 = "sha256-0F16o3oqOB5ri6KBdPFEFHB4dv1z+Pw6E5f1rwkqwi8="; rev = { revision = "4"; sha256 = "sha256-a5C58iYrL7eAEHCzinICiJpbNTGwiOFFAYik28et7fI="; }; } { }
That's a lot better than using overrideCabal
!
Hackage revisions in Nix
Today I got very confused when using callHackageDirect
to add the openapi3
package gave me errors like this
> Using Parsec parser > Configuring openapi3-3.2.3... > CallStack (from HasCallStack): > withMetadata, called at libraries/Cabal/Cabal/src/Distribution/Simple/Ut... > Error: Setup: Encountered missing or private dependencies: > base >=4.11.1.0 && <4.18, > base-compat-batteries >=0.11.1 && <0.13, > template-haskell >=2.13.0.0 && <2.20
When looking at its entry on Hackage those weren't the version ranges for the
dependencies. Also, running ghc-pkg list
told me that I already had all
required packages at versions matching what Hackage said. So, what's actually
happening here?
It took me a while before remembering about revisions but once I did it was
clear that callHackageDirect
always fetches the initial revision of a package
(i.e. it fetches the original tar-ball uploaded by the author). After realising
this it makes perfect sense – it's the only revision that's guaranteed to be
there and won't change. However, it would be very useful to be able to pick a
revision that actually builds.
I'm not the first one to find this, of course. It's been noted and written about
on the discource several years ago. What I didn't find though was a way to
influence what revision that's picked. It took a bit of rummaging around in the
nixpkgs
code but finally I found two variables that's used in the Hackage
derivation to control this
revision
- a string with the number of the revision, andeditedCabalFile
- the SHA256 of the modified Cabal file.
Setting them is done using the overrideCabal
function. This is a piece of my
setup for a modified set of Haskell packages:
hl = nixpkgs.haskell.lib.compose; hsPkgs = nixpkgs.haskell.packages.ghc963.override { overrides = newpkgs: oldpkgs: { openapi3 = hl.overrideCabal (drv: { revision = "4"; editedCabalFile = "sha256-a5C58iYrL7eAEHCzinICiJpbNTGwiOFFAYik28et7fI="; }) (oldpkgs.callHackageDirect { pkg = "openapi3"; ver = "3.2.3"; sha256 = "sha256-0F16o3oqOB5ri6KBdPFEFHB4dv1z+Pw6E5f1rwkqwi8="; } { });
It's not very ergonomic, and I think an extended version of callHackageDirect
would make sense.
Bending Warp
In the past I've noticed that Warp both writes to stdout
at times and produces
some default HTTP responses, but I've never bothered taking the time to look up
what possibilities it offers to changes this behaviour. I've also always thought
that I ought to find out how Warp handles signals.
If you wonder why this would be interesting to know there are three main points:
- The environments where the services run are set up to handle structured
logging. In our case it should be JSONL written to
stdout
, i.e. one JSON object per line. - We've decided that the error responses we produce in our code should be JSON, so it's irritating to have to document some special cases where this isn't true just because Warp has a few default error responses.
- Signal handling is, IMHO, a very important part of writing a service that runs well in k8s as it uses signals to handle the lifetime of pods.
Looking through the Warp API
Browsing through the API documentation for Warp it wasn't too difficult to find the interesting pieces, and that Warp follows a fairly common pattern in Haskell libraries
- There's a function called
runSettings
that takes an argument of typeSettings
. - The default settings are available in a variable called
defaultSettings
(not very surprising). There are several functions for modifying the settings and they all have the same shape
setX :: X -> Settings -> Settings.
which makes it easy to chain them together.
- The functions I'm interested in now are
setOnException
- the default handler,
defaultOnException
, prints the exception tostdout
using itsShow
instance setOnExceptionResponse
- the default responses are produced by
defaultOnExceptionResponse
and contain plain text response bodies setInstallShutdownHandler
- the default behaviour is to wait for all ongoing requests and then shut done
setGracefulShutdownTimeout
- sets the number of seconds to wait for ongoing requests to finnish, the default is to wait indefinitely
Some experimenting
In order to experiment with these I put together a small API using servant,
app
, with a main
function using runSettings
and stringing together a bunch
of modifications to defaultSettings
.
main :: IO () main = Log.withLogger $ \logger -> do Log.infoIO logger "starting the server" runSettings (mySettings logger defaultSettings) (app logger) Log.infoIO logger "stopped the server" where mySettings logger = myShutdownHandler logger . myOnException logger . myOnExceptionResponse
myOnException
logs JSON objects (using the logging I've written about before,
here and here). It decides wether to log or not using
defaultShouldDisplayException
, something I copied from defaultOnException
.
myOnException :: Log.Logger -> Settings -> Settings myOnException logger = setOnException handler where handler mr e = when (defaultShouldDisplayException e) $ case mr of Nothing -> Log.warnIO logger $ lm $ "exception: " <> T.pack (show e) Just _ -> do Log.warnIO logger $ lm $ "exception with request: " <> T.pack (show e)
myExceptionResponse
responds with JSON objects. It's simpler than
defaultOnExceptionResponse
, but it suffices for my learning.
myOnExceptionResponse :: Settings -> Settings myOnExceptionResponse = setOnExceptionResponse handler where handler _ = responseLBS H.internalServerError500 [(H.hContentType, "application/json; charset=utf-8")] (encode $ object ["error" .= ("Something went wrong" :: String)])
Finally, myShutdownHandler
installs a handler for SIGTERM
that logs and then
shuts down.
myShutdownHandler :: Log.Logger -> Settings -> Settings myShutdownHandler logger = setInstallShutdownHandler shutdownHandler where shutdownAction = Log.infoIO logger "closing down" shutdownHandler closeSocket = void $ installHandler sigTERM (Catch $ shutdownAction >> closeSocket) Nothing
Conclusion
I really ought to have looked into this sooner, especially as it turns out that
Warp offers all the knobs and dials I could wish for to control these aspects of
its behaviour. The next step is to take this and put it to use in one of the
services at $DAYJOB
Getting Amazonka S3 to work with localstack
I'm writing this in case someone else is getting strange errors when trying to use amazonka-s3 with localstack. It took me rather too long finding the answer and neither the errors I got from Amazonka nor from localstack were very helpful.
The code I started with for setting up the connection looked like this
main = do awsEnv <- AWS.overrideService localEndpoint <$> AWS.newEnv AWS.discover -- do S3 stuff where localEndpoint = AWS.setEndpoint False "localhost" 4566
A few years ago, when I last wrote some Haskell to talk to S3 this was enough1, but now I got some strange errors.
It turns out there are different ways to address buckets and the default, which
is used by AWS itself, isn't used by localstack. The documentation of
S3AddressingStyle
has more details.
So to get it to work I had to change the S3 addressing style as well and ended up with this code instead
main = do awsEnv <- AWS.overrideService (s3AddrStyle . localEndpoint) <$> AWS.newEnv AWS.discover -- do S3 stuff where localEndpoint = AWS.setEndpoint False "localhost" 4566 s3AddrStyle svc = svc {AWS.s3AddressingStyle = AWS.S3AddressingStylePath}
Footnotes:
That was before version 2.0 of Amazonka, so it did look slightly different, but overriding the endpoint was all that was needed.
Making Emacs without terminal emulator a little more usable
After reading Andrey Listopadov's You don't need a terminal emulator (mentioned
at Irreal too) I decided to give up on using Emacs as a terminal for my shell.
In my experience Emacs simply isn't a very good terminal to run a shell in
anyway. I removed the almost completely unused shell-pop from my configuration
and the keybinding with a binding to async-shell-command
. I'm keeping
terminal-here in my config for the time being though.
I realised projectile didn't have a function for running it in the root of a
project, so I wrote one heavily based on project-async-shell-command
.
(defun mep-projectile-async-shell-command () "Run `async-shell-command' in the current project's root directory." (declare (interactive-only async-shell-command)) (interactive) (let ((default-directory (projectile-project-root))) (call-interactively #'async-shell-command)))
I quickly found that the completion offered by Emacs for shell-command
and
async-shell-command
is far from as sophisticated as what I'm used to from Z
shell. After a bit of searching I found emacs-bash-completion. Bash isn't my
shell of choice, partly because I've found the completion to not be as good as
in Z shell, but it's an improvement over what stock Emacs offers. The
instructions in the repo was good, but had to be adjusted slightly:
(use-package bash-completion :straight (:host github :repo "szermatt/emacs-bash-completion") :config (add-hook 'shell-dynamic-complete-functions 'bash-completion-dynamic-complete))
I just wish I'll find a package offering completions reaching Z shell levels.