Posts tagged "nix":
Emacs via Nix with mu4e
I've been running development versions of Emacs ever since I switched to Wayland
and needed the PGTK code. The various X-git packages on AUR makes that easy,
as long as one doesn't mind building the packages locally, and regularly.
Building a large package like Emacs does get a bit tiring after a while though
so I started looking at the emacs overlay to see if I could keep up without
building quite that much.
The first attempt at this failed as I couldn't get my email setup working; emacs
simply refused to find the locally installed mu4e package. I felt I didn't
have time to solve it at the time, reverted back to doing the builds myself
again. It kept irritating me though, and today I made another attempt. This time
I invested a bit more time in reading up on how to install emacs via Nix with
packages. Something that paid off.
I'm managing my packages using nix profile and a flake.nix. To install emacs
with a working mu4e I started with adding the emacs overlay to the inputs
inputs = { nixpkgs.url = "github:nixos/nixpkgs?ref=nixpkgs-unstable"; ... community-emacs.url = "github:nix-community/emacs-overlay"; };
and in my outputs I made sure to use the overlay on nixpkgs
outputs = inputs@{ nixpkgs, community-emacs, ... }: let system = "x86_64-linux"; pkgs = import nixpkgs { inherit system; overlays = [ community-emacs.overlays.emacs ]; }; ...
and in the list of packages passed to pkgs.buildEnv I added
... ((emacsPackagesFor emacs-pgtk).emacsWithPackages (epkgs: [ epkgs.mu4e ])) mu ...
That's all there's to it. After running nix profile update 0 I had a build of
emacs with Wayland support that's less than a day old, all downloaded from the
community cache. Perfect!
Hoogle setup for local development
About a week ago I asked a question on the Nix Discourse about how to create a setup for Hoogle that
- includes the locally installed packages, and
- the package I'm working on, and ideally also
- have all local links, i.e. no links to Hackage.
I didn't get an answer there, but some people on the Nix Haskell channel on Matrix helped a bit, but it seems this particular use case requires a bit of manual work. The following commands get me an almost fully working setup:
cabal haddock --haddock-internal --haddock-quickjump --haddock-hoogle --haddock-html hoogle_dir=$(dirname $(dirname $(readlink -f $(which hoogle)))) hoogle generate --database=local.hoo \ $(for d in $(fd -L .txt ${hoogle_dir}); do printf "--local=%s " $(dirname $d); done) \ --local=./dist-newstyle/build/x86_64-linux/ghc-9.8.2/pkg-0.0.1/doc/html/pkg hoogle server --local --database=local.foo
What's missing is working links between the documentation of locally installed
packages. It looks like the links in the generated documention in Nix have a lot
of relative references containing ${pkgroot}/../../../../ which is what I
supect causes the broken links.
Nix, cabal, and tests
At work I decided to attempt to change the setup of one of our projects from using
to the triplet I tend to prefer
During this I ran into two small issues relating to tests.
hspec-discover both is, and isn't, available in the shell
I found mentions of this mentioned in an open cabal ticket and someone even made a git repo to explore it. I posted a question on the Nix discorse.
Basically, when running cabal test in a dev shell, started with nix develop,
the tool hspec-discover wasn't found. At the same time the packages was
installed
(ins)$ ghc-pkg list | rg hspec hspec-2.9.7 hspec-core-2.9.7 (hspec-discover-2.9.7) hspec-expectations-0.8.2
and it was on the $PATH
(ins)$ whereis hspec-discover hspec-discover: /nix/store/vaq3gvak92whk5l169r06xrbkx6c0lqp-ghc-9.2.8-with-packages/bin/hspec-discover /nix/store/986bnyyhmi042kg4v6d918hli32lh9dw-hspec-discover-2.9.7/bin/hspec-discover
The solution, as the user julm pointed out, is to simply do what cabal tells
you and run cabal update first.
Dealing with tests that won't run during build
The project's tests were set up in such a way that standalone tests and
integration tests are mixed into the same test executable. As the integration
tests need the just built service to be running they can't be run during nix
build. However, the only way of preventing that, without making code changes,
is to pass an argument to the test executable, --skip=<prefix>, and I believe
that's not possible when using developPackage. It's not a big deal though,
it's perfectly fine to run the tests separately using nix develop . command
.... However, it turns out developPackage and the underlying machinery is
smart enough to skip installing package required for testing when it's turned
off (using dontCheck). This is the case also when returnShellEnv is true.
Luckily it's not too difficult to deal with it. I already had a variable
isDevShell so I could simply reuse it and add the following expression to
modifier
(if isDevShell then hl.doCheck else hl.dontCheck)
Update to Hackage revisions in Nix
A few days after I published Hackage revisions in Nix I got a comment from
Wolfgang W that the next release of Nix will have a callHackageDirect with
support for specifying revisions.
The code in PR #284490 makes callHackageDirect accept a rev argument. Like
this:
haskellPackages.callHackageDirect { pkg = "openapi3"; ver = "3.2.3"; sha256 = "sha256-0F16o3oqOB5ri6KBdPFEFHB4dv1z+Pw6E5f1rwkqwi8="; rev = { revision = "4"; sha256 = "sha256-a5C58iYrL7eAEHCzinICiJpbNTGwiOFFAYik28et7fI="; }; } { }
That's a lot better than using overrideCabal!
Hackage revisions in Nix
Today I got very confused when using callHackageDirect to add the openapi3
package gave me errors like this
> Using Parsec parser > Configuring openapi3-3.2.3... > CallStack (from HasCallStack): > withMetadata, called at libraries/Cabal/Cabal/src/Distribution/Simple/Ut... > Error: Setup: Encountered missing or private dependencies: > base >=4.11.1.0 && <4.18, > base-compat-batteries >=0.11.1 && <0.13, > template-haskell >=2.13.0.0 && <2.20
When looking at its entry on Hackage those weren't the version ranges for the
dependencies. Also, running ghc-pkg list told me that I already had all
required packages at versions matching what Hackage said. So, what's actually
happening here?
It took me a while before remembering about revisions but once I did it was
clear that callHackageDirect always fetches the initial revision of a package
(i.e. it fetches the original tar-ball uploaded by the author). After realising
this it makes perfect sense – it's the only revision that's guaranteed to be
there and won't change. However, it would be very useful to be able to pick a
revision that actually builds.
I'm not the first one to find this, of course. It's been noted and written about
on the discource several years ago. What I didn't find though was a way to
influence what revision that's picked. It took a bit of rummaging around in the
nixpkgs code but finally I found two variables that's used in the Hackage
derivation to control this
revision- a string with the number of the revision, andeditedCabalFile- the SHA256 of the modified Cabal file.
Setting them is done using the overrideCabal function. This is a piece of my
setup for a modified set of Haskell packages:
hl = nixpkgs.haskell.lib.compose; hsPkgs = nixpkgs.haskell.packages.ghc963.override { overrides = newpkgs: oldpkgs: { openapi3 = hl.overrideCabal (drv: { revision = "4"; editedCabalFile = "sha256-a5C58iYrL7eAEHCzinICiJpbNTGwiOFFAYik28et7fI="; }) (oldpkgs.callHackageDirect { pkg = "openapi3"; ver = "3.2.3"; sha256 = "sha256-0F16o3oqOB5ri6KBdPFEFHB4dv1z+Pw6E5f1rwkqwi8="; } { });
It's not very ergonomic, and I think an extended version of callHackageDirect
would make sense.
Patching in Nix
Today I wanted to move one of my Haskell projects to GHC 9.2.4 and found that
envy didn't compile due to an upper bound on its dependency on bytestring, it
didn't allow 0.11.*.
After creating a PR I decided I didn't want to wait for upstream so instead I
started looking into options for patching the source of a derivation of a
package from Hackage. In the past I've written about building Haskell packages
from GitHub and an older one were I used callHackageDirect to build Haskell
packages from Hackage. I wasn't sure how to patch up a package from Hackage
though, but after a bit of digging through haskell-modules I found appendPatch.
The patch wasn't too hard to put together once I recalled the name of the patch
queue tool I used regularly years ago, quilt. I put the resulting patch in the
nix folder I already had, and the full override ended up looking like this
... hl = haskell.lib; hsPkgs = haskell.packages.ghc924; extraHsPkgs = hsPkgs.override { overrides = self: super: { envy = hl.appendPatch (self.callHackageDirect { pkg = "envy"; ver = "2.1.0.0"; sha256 = "sha256-yk8ARRyhTf9ImFJhDnVwaDiEQi3Rp4yBvswsWVVgurg="; } { }) ./nix/envy-fix-deps.patch; }; }; ...
Simple nix flake for Haskell development
Recently I've moved over to using flakes in my Haskell development projects. It took me a little while to arrive at a pattern a flake for Haskell development that I like. I'm hoping sharing it might help others when doing the same change
{ inputs = { nixpkgs.url = "github:nixos/nixpkgs"; flake-utils.url = "github:numtide/flake-utils"; }; outputs = { self, nixpkgs, flake-utils }: flake-utils.lib.eachDefaultSystem (system: with nixpkgs.legacyPackages.${system}; let t = lib.trivial; hl = haskell.lib; name = "project-name"; project = devTools: # [1] let addBuildTools = (t.flip hl.addBuildTools) devTools; in haskellPackages.developPackage { root = lib.sourceFilesBySuffices ./. [ ".cabal" ".hs" ]; name = name; returnShellEnv = !(devTools == [ ]); # [2] modifier = (t.flip t.pipe) [ addBuildTools hl.dontHaddock hl.enableStaticLibraries hl.justStaticExecutables hl.disableLibraryProfiling hl.disableExecutableProfiling ]; }; in { packages.pkg = project [ ]; # [3] defaultPackage = self.packages.${system}.pkg; devShell = project (with haskellPackages; [ # [4] cabal-fmt cabal-install haskell-language-server hlint ]); }); }
The main issue I ran into is getting a development shell out of
haskellPackages.developPackage, it requires returnShellEnv to be true.
Something that isn't too easy to find out. This means that the only solution
I've found to getting a development shell is to have separate expressions for
building and getting a shell. In the above flake the build expression, [3],
passes an empty list of development tools, the argument devTools at [1],
while the development shell expression, [4], passes in a list of tools needed
for development only. The decision of whether the expression is for building or
for a development shell, [2], then looks at the list of development tools
passed in.
ZSH, Nix, and completions
TIL that ZSH completions that come with Nix packages end up in
~/.nix-profile/share/zsh/vendor-completions/ and that folder is not added to
$FPATH by the init script that comes with Nix.
After modifying the bit in ~/.zshenv it now looks like this
if [[ -f ~/.nix-profile/etc/profile.d/nix.sh ]]; then source ~/.nix-profile/etc/profile.d/nix.sh export fpath=(~/.nix-profile/share/zsh/vendor-completions ${fpath}) fi
First contribution to nixpkgs.haskellPackages
Nothing much to be proud of, but yesterday I found out that servant-docs was marked broken in nixpkgs even though it builds just fine and this morning I decided to do something about it.
So, with the help of a post on the NixOS discourse I put together my first PR.
Nix shell, direnv and XDG_DATA_DIRS
A few weeks ago I noticed that I no longer could use
haskell-hoogle-lookup-from-website in Emacs. After a bit of experimentation I
found that the reason was that I couldn't use xdg-open in a Nix shell.
Yesterday I finally got around to look into further.
It's caused by direnv overwriting XDG_DATA_DIRS rather than appending to it.
Of course someone already reported a bug already.
The workaround is to use
use nix --keep XDG_DATA_DIRS
Haskell, Nix and using packages from GitHub
The other day I bumped into what turned out to be a bug in Amazonka where
sockets weren't closed in a timely fashion and thus the process ran out of file
descriptors. Some more digging and an issue later I found that a fix most likely
already in place (mine was possibly a duplicate of an older issue). Now I only
had to verify if that was the case by using the most recent, and unreleased code
on the develop branch of Amazonka.
My first thought was to attempt to instruct Cabal to build the bits of Amazonka
I need by putting a few source-repository-package stanzas in my config. That
quickly started to look like a bit of a rabbit hole, so I decided to use Nix
instead. After finding the perfect SO post and looking up yet again how to do
overrides for Haskell I ran cabal2nix for the three packages I need:
cabal2nix --no-haddock --no-check --subpath amazonka \ git://github.com/brendanhay/amazonka.git > amazonka.nix cabal2nix --no-haddock --no-check --subpath core \ git://github.com/brendanhay/amazonka.git > amazonka-core.nix cabal2nix --no-haddock --no-check --subpath amazonka-sqs \ git://github.com/brendanhay/amazonka.git > amazonka-sqs.nix
The relevant part of the old Nix expression looked like this:
thePkg = haskellPackages.developPackage { root = lib.cleanSource ./.; name = name; modifier = (t.flip t.pipe) [hl.dontHaddock hl.enableStaticLibraries hl.justStaticExecutables hl.disableLibraryProfiling hl.disableExecutableProfiling]; };
After adding the overrides it looked like this
hp = haskellPackages.override { overrides = self: super: { amazonka-core = self.callPackage ./amazonka-core.nix {}; amazonka = self.callPackage ./amazonka.nix {}; amazonka-sqs = self.callPackage ./amazonka-sqs.nix {}; }; }; thePkg = hp.developPackage { root = lib.cleanSource ./.; name = name; modifier = (t.flip t.pipe) [hl.dontHaddock hl.enableStaticLibraries hl.justStaticExecutables hl.disableLibraryProfiling hl.disableExecutableProfiling]; };
After a somewhat longer-than-usual build I could verify that I had indeed bumped into the same issue and my issue was a duplicate.
Better Nix setup for Spacemacs
In an earlier post I documented my setup for getting Spacemacs/Emacs to work with Nix. I've since found a much more elegant solution based on
- direnv, and
- emacs-direnv
No more Emacs packages for Nix and no need to defining functions that wrap
executables in an invocation of nix-shell.
There's a nice bonus too, with this setup I don't need to run nix-shell, which
always drops me at a bash prompt, instead I get a working setup in my shell of
choice.
Setting up direnv
The steps for setting up direnv depends a bit on your setup, but luckily I
found the official instructions for installing direnv to be very clear and
easy to follow. There's not much I can add to that.
Setting up Spacemacs
Since emacs-direnv isn't included by default in Spacemacs I needed to do a bit
of setup. I opted to create a layer for it, rather than just drop it in the list
dotspacemacs-additional-packages. Yes, a little more complicated, but not
difficult and I nurture an intention of submitting the layer for inclusion in
Spacemacs itself at some point. I'll see where that goes.
For now, I put the following in the file
~/.emacs.d/private/layers/direnv/packages.el:
(defconst direnv-packages '(direnv)) (defun direnv/init-direnv () (use-package direnv :init (direnv-mode)))
Setting up the project folders
In each project folder I then add the file .envrc containing a single line:
use_nix
Then I either run direnv allow from the command line, or run the
function direnv-allow after opening the folder in Emacs.
Using it
It's as simple as moving into the folder in a shell – all required envvars are set up on entry and unset on exit.
In Emacs it's just as simple, just open a file in a project and the envvars are set. When switching to a buffer outside the project the envvars are unset.
There is only one little caveat, nix-build doesn't work inside a Nix shell. I
found out that running
IN_NIX_SHELL= nix-build
does work though.
My ghcide build for Nix
I was slightly disappointed to find out that not all packages on Hackage that are marked as present in Nix(pkgs) actually are available. Quite a few of them are marked broken and hence not installable. One of these packages is ghcide.
There are of course expressions available for getting a working ghcide
executable installed, like ghcide-nix. However, since I have rather simple needs
for my Haskell projects I thought I'd play with my own approach to it.
What I care about is:
- availability of the development tools I use, at the moment it's mainly
ghcidebut I'm planning on making use of ormolu in the near future - pre-built packages
- ease of use
So, I put together ghcide-for-nix. It's basically just a constumized Nixpkgs
where the packages needed to un-break ghcide are present.
Usage is a simple import away:
import (builtins.fetchGit { name = "ghcide-for-nix"; url = https://github.com/magthe/ghcide-for-nix; rev = "927a8caa62cece60d9d66dbdfc62b7738d61d75f"; })
and it'll give you a superset of Nixpkgs. Pre-built packages are available on Cachix.
It's not sophisticated, but it's rather easy to use and suffices for my purposes.
Nix setup for Spacemacs
Edit 2020-06-22: I've since found a better setup for this.
When using ghcide and LSP, as I wrote about in my post on Haskell, ghcide, and
Spacemacs, I found myself ending up recompiling a little too often. This pushed
me to finally start looking at Nix. After a bit of a fight I managed to
get ghcide from Nix,
which brought me the issue of setting up Spacemacs. Inspired by a gist from
Samuel Evans-Powell and a guide to setting up an environment for Reflex by
Thales Macedo Garitezi I ended up with the following setup:
(defun dotspacemacs/layers () (setq-default ... dotspacemacs-additional-packages '( nix-sandbox nix-haskell-mode ... ) ... ))
(defun dotspacemacs/user-config () ... (add-hook 'haskell-mode-hook #'lsp) (add-hook 'haskell-mode-hook 'nix-haskell-mode) (add-hook 'haskell-mode-hook (lambda () (setq-local flycheck-executable-find (lambda (cmd) (nix-executable-find (nix-current-sandbox) cmd))) (setq-local flycheck-command-wrapper-function (lambda (argv) (apply 'nix-shell-command (nix-current-sandbox) argv))) (setq-local haskell-process-wrapper-function (lambda (argv) (apply 'nix-shell-command (nix-current-sandbox) argv))) (setq-local lsp-haskell-process-wrapper-function (lambda (argv) `("nix-shell" "-I" "." "--command" "ghcide --lsp" ,(nix-current-sandbox)))))) (add-hook 'haskell-mode-hook (lambda () (flycheck-add-next-checker 'lsp-ui '(warning . haskell-stack-ghc)))) ... )
It seems to work, but please let me know if you have suggestions for improvements.