Getting a Raspberry Pi 5

2025-07-19T00:00:00+00:00
I bought a Raspberry Pi 5 because I wanted to run a temperature sensor. Mostly so I can check how effective my portable air conditioning unit from DeLonghi actually is in my apartment.
I naively assumed that v5 of the Pi (as opposed to my ancient v3) would support video over USB-C, or at least have a standard USB-C port. I also assumed I could just flash Home Assistant onto a USB stick and it would magically install itself onto the SSD I got (together with the M.2 HAT).
My assumptions were all wrong.
It has a micro HDMI port (didn’t know that existed), USB-C is for power, and, as far as I can tell, plugging in a USB stick won’t do anything.
So now I ordered a cable, a SD card, an SD card adapter, and I also added a SD card reader for good measure. I now expect NOTHING to work, so I won’t assume that my MacBook Pro’s SD card reader will work with these cards. Hence the USB pluggable SD card reader.
I’m sure the Pi is great, but right now I just want to throw it in the trash.
The idea is now to copy the standard Raspberry Pi image onto the SD card and boot the Pi from that. Then SSH into it and use the imager tool to install Home Assistant onto the SSD card. Finally, change the boot order so the SSD card comes first.
In the end I couldn’t wait for my SD card to arrive. I had to re-assemble my old desktop computer anyway, before I could sell it. So I used the NVMe slot on its mainboard to plugin the Rasperry Pi NVMe SSD, and used rpi-imager</code> to flash Home Assistant onto it.
The Pi immediately booted from the NVMe, without having to change anything in the boot order.
It then took me a while to realize that my old, crappy ethernet cable was constantly wiggling itself loose, resulting in a lost connection to the HA web interface.
Then I tried to setup my ConBee 3 stick. The web updater gave me an error (of course), and the GCFFlasher</code> tool in NixOS is some minor versions behind and also didn’t work. It took me a while to realize that that was the case. I then had to compile it from scratch. devenv</a> came in very handy. I was in a hurry and had no intention to figure out how to setup a C environment. So I just did devenv init</code> and set languages.cplusplus.enable</code> to true. Worked flawlessly.
Now my sensor is finally connected. It’s 27 degrees Celsius in my bed room, apparently.
What an adventure.


Advent of Code 2024 Day 6: Janet Streams Using Fibers/Coroutines
2024-12-07T00:00:00+00:00
Remember when it was all the rage to write articles about functional programming (FP) in Javascript by explaining how you can chain .map</code>, .filter</code> and .reduce</code> on lists? I hated that. I think that it conditions people into thinking that FP is all about list comprehensions. And when lists aren’t convenient anymore (more on this later), they then resort to imperative programming, thinking that FP isn’t suited to that kind of task.</p>
In Advent of Code (AoC) there are plenty of examples where eager list comprehensions won’t get you very far. An innocent looking map().filter().map()</code> can consume all your memory and make the garbage collector go crazy if you are creating and re-creating huge lists. A straight forward solution, as I mentioned earlier, is to fall back to more imperative patterns, such as the loop</code> macro in Janet.</p>
I was hoping that I could use Janet’s fibers/coroutines in combination with its stdlib map</code> and filter</code> function to work on streams rather than eagerly evaluated lists. But that was not the case:</p>
(->> (gen-stuff input)
     (map foo)
     (filter bar))
</code></pre>
In my testing on AoC day 6 (the first one where you walk around in a grid), the imperative version using loop</code> used about 80MB, the version that kicks things off with a generator and then runs this through various list comprehenions immediately took up around 10GB of memory. Here’s a concrete example:</p>
(defn gen-nums [] (coro (for i 0 10000000 (yield i)))

(comment
  # low mem usage
  (each v (gen-nums) (print v))

  # high mem usage
  (each v (map |(string/repeat (string $) 100) (gen-nums)) (print v)))
</code></pre>
But it turns out that it is surprisingly straight forward to have your cake and eat it too. You can use easily define streaming versions of map</code> and filter</code> using the coro</code> function (or macro?):</p>
(defn map* [f ds] (coro (each v ds (yield (f v)))))

(defn filter* [f ds] (coro (each v ds (when (f v) (yield v)))))
</code></pre>
I’ve used these for an upated implementation of day 6, which you can find here</a></p>


Advent of Code 2024 Day 2: Parsing Expression Grammars
2024-12-02T08:08:55+00:00
I am fully committed to using Janet this year! At least I hope so. One of my goals is to get really familiar and fluent with using parsing expression grammars (PEG). I’ve always liked parser combinators, which seem to be more or less the same thing. One of the most basic and typical ways to parse input in Advent of Code is to split a string into a lines and split each line on spaces.</p>
1 2 3
4 5 6
</code></pre>
This an be achieved quite elegantly with a somewhat more recent addition to the PEG library, split</code>. You give it two patterns, one for the separator and one for the actual contents, and it generally does the right thing. There is just one caveat. In the following snippet, there’s a space at the end of my input string. If you remove the asterisk *</code> from the digit pattern, you get no matches. So split</code> more or less requires you to make the content optional, unless you are absolutely certain that you won’t have a dangling separator at the end of your input.</p>
  (peg/match ~(split :s ':d*) "1 2 3 ")
</code></pre>
By making the second pattern optional, the end of the string, “3 “ can now be matched as two digits separated by a space. The second digit is simply empty. Sounds totally logical, I know. But you probably don’t want an empty capture. You can either remove it later through filter</code> or you can call string/trimr</code> on the input, which is my convention for AoC. This removes trailing whitespace, so you no longer need to make the second pattern optional, like this:</p>
(peg/match ~(split :s ':d) (string/trimr "1 2 3 "))
</code></pre>
Of course you’ll have to adjust your code for other separators (or filter</code> it out later).</p>
Anyway, back to the real world (of AoC). Here’s the same PEG with and without using split</code>. I think it’s a huge improvement.</p>
# before
(def input-peg
  (peg/compile
    ~{:main (* (some (* :line (? :s))) -1)
      :line (group (some (* :num (any " "))))
      :num (/ (<- :d+) ,scan-number)}))

# after
(def parser
  (peg/compile ~(split "\n" (group (split :s (any (number :d+)))))))
</code></pre>


Dark/Light Switching for the Terminal
2024-10-09T00:00:00+00:00
I firmly believe that you should use light mode when the ambient lighting is bright. As far as I know, science agrees with that, since dark text on a white background is easier to read than the opposite.</p>
But after the sun has set and you’ve dimmed all the lights, staring at a bright screen seems wrong and it looks out of place when the rest of your system switches to dark mode.</p>
For the longest time I’ve therefore wanted to have the automatic dark/light mode switching in my terminal environment, which consists of Alacritty, tmux, Fish and Neovim (tmux doesn’t define any color values of its own, so we can ignore it).</p>
I finally got around to spending some time on this not so tricky problem and I’d say I’m 85% there. I use my own color theme</a> which exports config files for, among other things, all three programs listed above. And all config files are available in a dark and a light mode. Meaning, in Neovim I can simply switch to the dark version with :color yui_dark</code>. Since Neovim has a nice client/server architecture I can also do this remotely:</p>
:let g:lightline.colorscheme = "yui_dark"
    \ | colorscheme yui_dark
    \ | call lightline#init()
    \ | call lightline#colorscheme()
    \ | call lightline#update()<CR>
</code></pre>
Alacritty also has a nice API for changing config values from the shell. You can read more about it under man alacritty-msg</code> (and man 5 alacritty</code> for the config values) but the gist is alacritty msg config -w -1 'colors.cursor.background="#CCCCCC"'</code>.</p>
Lastly, for configuring the Fish shell syntax colors you use shell commands anyway, so the exported “config” is a Fish file that I can call whenever I want. The file has commands like this:</p>
set fish_color_match eae9e9
set fish_color_selection --background=474355
set fish_color_search_match --background=4f4b5f
...
</code></pre>
In other words: I now have shell commands for remotely changing Alacritty, Fish and Neovim from light to dark mode and vice versa. The “only” thing left to do is to find a way of running either of the two variants when the system switches themes.</p>


Heist: Haskell Templating Woes
2023-08-31T00:00:00+00:00
Yesterday I got my feet wet by rendering a single, measly splice. And at first it didn’t look like I’d even accomplish that before the end of the day. Today I want to extend the example by adding more splices that operate on different data. In the compiled splices tutorial</a> they have an example that shows how to render a list of persons. But in a real world application with dozens of routes you will have dozens of splices that all require different data. But if all of those compiled splices end up in your Heist state under a single type, would that type end up being the concatentation of all the parameters of all splices? Let’s find out.</p>
The goal for today is this: Make a (fake) database call and use the data to generate two splices, one for a Text</code> and one for an Int</code>, that are then used in a template.</p>
I specifically do not want to create some application monad and access it from within splices. My major concern with this is that individual splices are now free to make database calls, much like a GraphQL API where each field is backed by an independent resolver, which can make as many database calls as it wants. I’m actually not a fan of this application monad pattern, because it makes it all too easy to have code access your logger or your database that really shouldn’t.</p>
Here’s how I imagine this will work in pseudo-code:</p>
main = do
  data <- fakeDatabaseCall
  heistState <- initHeist { ... }

  let spliceA = genSpliceA (data.someNumber)
  let spliceB = genSpliceB (data.someText)

  heistState.splices = heistState.splices
    // { spliceA = spliceA, spliceB = spliceB }

  renderTemplate heistState "foo"
</code></pre>
I suspect that merging splices into the Heist state will be a major undertaking. I have no idea how I can modify Heist state. Indeed, the documentation for initHeist</code> says this:</p>

We don’t provide functions to add either type of loadtime splices to your HeistState after initHeist because it doesn’t make any sense unless you re-initialize all templates with the new splices.</p>
</blockquote>
The documentation does mention</p>

Heist’s HeistState -> HeistState “filter” functions.</p>
</blockquote>
but I don’t know where they are. They do have a few functions that all work in the HeistT n m ()</code> environment, but I don’t know how I would use them. I could call modifyHS</code> in a splice function, but then I’m once again in a load-time splice function and I want to avoid the whole runtime stuff-everything-into-an-application monad.</p>
So I guess modifying the Heist state is not an option after all. What else can I do then, that’s not the giant-all-encompassing-application-monad-of-doom? The tutorial code has this snippet in it:</p>
hs <- load baseDir ("people" ## allPeopleSplice)
let runtime = fromJust $ C.renderTemplate hs "people"
builder <- evalStateT (fst runtime)
            [ Person "John" "Doe" 42
            , Person "Jane" "Smith" 21
            ]
return $ toByteString builder
</code></pre>
Notice that they’re supplying the data for this view through a simple, hard coded state monad. I could create runtime splices that use a bespoke reader monad and then in my route handlers fill that reader monad with all the data for that route. I suspect that this will hamper re-use though. If you have a splice that needs an int and its used in two different templates that use a different reader monad each, then I can think of two options:</p>

duplicate the splice</li>
make the splice a bit more generic with a type class and then implement that type class for each monad</li>
</ul>
class HasNewsCount a where
  getNewsCount :: a -> Int

instance HasNewsCount ViewA where
  getNewsCount = undefined

instance HasNewsCount ViewB where
  getNewsCount = undefined
</code></pre>
Anyway, let’s give this a try. First, I need a somewhat more realistic example. I created two views for two routes:</p>
// view_a.tpl
<apply template="index">
  <person />
  <foo />
</apply>

// view_b.tpl
<apply template="index">
  <count />
  <foo />
</apply>
</code></pre>
They both have <foo /></code> in common, so that I can go through the use case of having a shared splice that gets its data from different reader monads (unless I figure out a solution that doesn’t need those monads, but I doubt it). They also each have a splice that’s exclusive to the view, so that the data for each view as a whole is different.</p>
I cleaned up main.hs</code> which now looks like this:</p>
mainSplices :: Monad m => Splices (C.Splice m)
mainSplices = return mempty

main :: IO ()
main = do
  let spliceConfig =
        mempty
          & scLoadTimeSplices .~ defaultLoadTimeSplices
          & scTemplateLocations .~ [loadTemplates "app"]

  eitherHeistState <-
    initHeist $
      emptyHeistConfig
        & hcNamespace .~ ""
        & hcErrorNotBound .~ False
        & hcSpliceConfig .~ spliceConfig
        & hcCompiledSplices .~ mainSplices

  case eitherHeistState of
    Left err ->
      putStrLn $ "Heist init failed: " ++ show err
    Right heistState -> do
      case C.renderTemplate heistState "view_a" of
        Nothing -> do
          putStrLn "Index not found!"
        Just (docRuntime, _) -> do
          docBuilder <- docRuntime
          print $ toByteString docBuilder
</code></pre>
I’m already scared of this Monad m => Splices (C.Splice m)</code> being my downfall. This m</code> will have to be specialized to an appropriate reader monad for each view. But without having it be the amalgamation of all views (or their data).</p>
And just as I thought, this doesn’t work. In retrospect it’s rather obvious.</p>
fooSplice :: (MonadIO m, MonadReader e m, HasFoo e) => C.Splice m
fooSplice = do
  return $ C.yieldRuntimeText $ do
    fooValue <- lift $ asks foo
    return $ T.pack $ show fooValue
</code></pre>
If you build a few splices like that, all with a different Has*</code> constraint, the top level splices list must gather up all those constraints.</p>
Recapping:</p>

modifying the list of splices after initHeist</code> seems hard and not what you’re supposed to do</li>
any concrete type in a splice will bubble up to the top level splices definition, where you’ll be forced to have a type that is the concatenation of all child types</li>
</ul>
I took another look at the Stack Overflow answer</a> here and specifically looked at the source code mentioned in a comment. But that source code seems to reference a Heist version that is practically ancient at this point. I don’t think the option mentioned by “mightybyte” is viable anymore.</p>
I updated the example repository at this commit</a>.</p>
Next I’ll have to figure out which other templating library I can use or if I want to just drop Haskell for this project.</p>


Two Years of Nix & Home Manager
2023-06-05T00:00:00+00:00
Summary</h2>
After utilizing Nix and HM to manage both a MacOS and a NixOS
machine for a solid two years, I can confidently say that the experience
has been incredibly pleasant overall. Reflecting back, the major
advantages that stood out, and continue to do so, include:</p>

Seamless project environment management with
nix-direnv</code></a>, which, in
my opinion, is unmatched.</li>
Having a single package manager instead of one for each application.
This not only simplifies package management but also enables upgrading
everything with just a single command. Nix has effectively replaced
the following plugin/package managers for me:

Neovim plugins</li>
Fish shell plugins</li>
System packages (for example AUR, MacPorts)</li>
tmux plugins</li>
Language package managers (npm, cargo, and so on)1</a></sup></li>
Visual Studio Code plugins</li>
</ul>
</li>
</ul>
At one point, I briefly reverted back to using only MacPorts and a basic
dotfiles repository. I questioned whether the added complexity of Nix
and HM was truly worth it. But without the integration between
nix-direnv</code> and my shell, switching between projects became a chore.
Upgrading packages now meant interacting with multiple, different upgrade
processes instead of just a single command. Plus, if I ever decide
to upgrade my desktop computer and go back to Linux, I’ll have to
painstakingly sync both setups manually.</p>
If you’re content with your current setup, then the effort of learning
Nix might not outweigh its benefits.</p>
However, if you see room for improvement, I strongly recommend exploring
Nix.</p>
What worked well</h2>
Unparalleled Reliability</h3>
If you’re someone who enjoys tinkering with Linux, then in my opinion,
there’s simply no better alternative to NixOS. The ability to replace
your boot loader, shell, Kernel parameters, and practically anything you
want, reboot, and effortlessly revert back to the previous state of
your system is an absolute superpower. It eradicates any fear of having
to troubleshoot issues from a rescue USB stick, thereby empowering you
to boldly experiment. The same level of confidence extends to system
updates as well. I can’t recall a single instance where I was unable to
use my NixOS system.</p>
Another aspect I appreciate is how NixOS configuration options often go
beyond merely mirroring the configuration of the underlying software in
a one-to-one manner. Take, for instance, enabling wireplumber, which not
only modifies files in /etc/</code> but also adds a
DBUS_SESSION_BUS_ADDRESS</code> environment variable, and so on. Another
example is enabling bspwm</code>, a tiling window manager. Setting just
bspwm.enable = true;</code> will install the package and add xserver
configuration</a>
that also gets rid of some annoying issues in certain Java GUI
applicatoins. This relieves me from meticulously keeping track of
interactions between various services and tools. NixOS generally strikes
a good balance between making these options genuinely useful while not
having too many unexpected consequences. Ultimately, the ability to
enable something and have it reliably do the right thing is something I
really came to appreciate.</p>
Effortless Configuration Sharing</h3>
Sharing configuration between MacOS and NixOS has been very straight
forward overall. My approach is to have a .nix</code> file to consolidate
shared packages and configuration. This file is then imported into the
host-specific home.nix</code> file, which can include additional packages and
configuration specific to each host. To illustrate, here are two code
snippets demonstrating how to integrate tmux with the system clipboard
on each respective platform. Both snippets are merged with the shared
tmux configuration (in this case by simply appending the strings to the
end of the shared configuration).</p>
# NixOS
{
  programs.tmux.extraConfig = ''
    bind -T copy-mode-vi y send-keys -X copy-pipe-and-cancel "${pkgs.xsel}/bin/xsel -i --clipboard"
  '';
}
</code></pre>
# MacOS
{
  programs.tmux.extraConfig = ''
    bind-key -T copy-mode-vi y send -X copy-pipe-and-cancel "pbcopy"
  '';
}
</code></pre>
Project Environments With nix-direnv</code></h3>
One of the features I really missed when attempting to go back to a
simpler setup was the ability to seamlessly switch between different
work repositories. Each repository often required a distinct set of
tools, such as varying NodeJS versions, language toolchains, database
clients, S3 tools, and so on. Even in 2023, it appears that efficiently
managing this is still an unsolved mystery</del> problem.</p>
You can put all of these things in a Docker container but to be honest,
I have never found developing through a Docker container a pleasant
experience. So in the end, at last at $DAY_JOB, we have Docker
containers to run the actual software either locally or in production,
but the development environment itself is something every developer
handles differently. Most people rely on a mix of global tools that
hopefully don’t conflict and a gazillion version managers.</p>
What I do is create a repository that has a single flake.nix</code> file. In
that file, I define “development shells” for each project, like so:</p>
{
  shared = [
    google-cloud-sdk
    kubectl # pinned to a specific version
    aws-mfa
  ];

  project1 = {
    buildInputs = [
      go
      go-migrate pgformatter
      gopls
      go-tools
      golangci-lint
      go-outline
      gopkgs
      delve
    ];
  };

  project2 = {
    buildInputs = [
      nodePackages.typescript-language-server
      nodePackages.prettier
    ];
  };
}
</code></pre>
In every project directory, I then add a .envrc</code> file, where I tell
nix-direnv</code> to use one of those development
environments</a> with
a simple one-liner like this: use flake /path/to/flake#project</code>. I can also
put other code in that .envrc</code> file2</a></sup>, but that’s not really different from
using the non-nix version of direnv</code>. Whenever I enter one of those
directories, my shell magically has the tools I specified for that project
available on PATH</code>. This works with different shells (I use Fish for example).</p>
In other words:</p>
$ which go
$ cd my_project/
$ which go
/nix/store/rhmf86rrq5sksqhg1544dq6hvxrr5cvg-go-1.20.4/bin/go
$ cd ~
$ which go
</code></pre>
I can not live without this feature anymore. The combination of
effortlessly having per-project tools and being able to maintain those
environments in a different repository if needed (so your coworkers
don’t need to know about Nix), is just so good! Even if you don’t want
to use nix-direnv</code> I really encourage you to look at the non-Nix
version of direnv</code></a>!</p>
Simple but not Easy</h3>
According to Rich Hickey</a>,
the distinction between “easy” and “simple” lies in the level of effort
and complexity involved. “Easy” implies something intuitive and
effortless initially but can lead to increasing complexity in the long
run. On the other hand, “simple” requires more upfront investment but
results in overall reduced complexity. This is how I’ll be using these
terms in this chapter.</p>
Recently, I contributed two packages to the MacPorts package repository.
While the onboarding process was considerably easier compared to Nix, it
didn’t take long until I encountered issues related to leftover build
artifacts. Each build would yield different output, with the first build
often returning errors that didn’t appear in subsequent ones. To address
this, I resorted to deleting certain folders containing build-related
files, hoping to restore a clean build environment. In other words, it
was easy to get started but difficult to reach a comprehensive
understanding of what’s actually going on.</p>
With Nix on the other hand, the first steps are anything but intuitive.
Nix is a functional programming language, which in itself can be quite
demanding. Because it is used in so many different areas3</a></sup>,
documentation is often abstract. There’s no such thing as a “definitive
10-step guide for managing your home folder with Nix” in the official
manual. Compare this to MacPorts, which has a much narrower focus, and
as such the documentation can be a lot more specific and hands-on.</p>
However, Nix also embodies a sense of simplicity: same input, same
output. This concept is tremendously powerful and empowering. It enables
a highly motivating edit-compile feedback loop. I believe it is this
simplicity that led me to contribute to Nixpkgs, making it the first
package repository I ever contributed to.</p>
Undoubtedly, certain aspects of Nix can be challenging. For instance,
dealing with dependencies in JavaScript and Python applications can be
incredibly messy. I’ve come to really hate Javascript’s postInstall</code>
scripts, which serve as an escape hatch allowing arbitrary actions
during a build. They are often used to install binaries that are then
called by the Javascript code of that package. Consequently, some
packages incorporate a makeshift, ad-hoc package manager that determines
the platform, downloads a binary, and places it in the hopefully correct
location. However Nix doesn’t allow network requests in its build
sandbox. Therefore, getting such Javascript packages to play nicely with
Nix can be a challenge.</p>
And that’s just one example. In general, the more complex and messy the
packaging process, the greater the difficulty in making it work
seamlessly with Nix.</p>
But I believe that the eventual simplicity of Nix is often (not always)
worth the initial cost.</p>
Streamlined Package Management</h3>
When I switched from Nix to MacPorts, it became evident how many package
managers I relied on:</p>

MacPorts</li>
Fish plugin manager</li>
Vim/Neovim plugin manager</li>
Visual Studio Code plugin manager</li>
tmux plugin manager</li>
NPM or Yarn</li>
</ul>
However, with Nix and HM, I can consolidate all of these into
just Nix. This means I have a single command to update all my packages
across all my tools, a single command to install everything on a new
machine, and a unified set of concepts and knowledge applicable to all
my packaging needs.</p>
How much this matters to you depends on how many package and plugin
managers you use right now. How well they work, how often you need to
fiddle with what they do. Many of my colleagues are content with using
an IntelliJ IDE and Homebrew, and that works perfectly well for them.</p>
There is one aspect of this I want to emphasize. Even though Nixpkgs is already
one of the biggest package repositories out there, I occasionally come across
missing packages. For instance, as of writing this article,
ojroques/nvim-lspfuzzy</code></a> is not
part of Nixpkgs. In the absence of Nix, I would have resorted to manually
cloning the repository and placing the files in the appropriate directories (or
install a Neovim plugin manager). However, with Nix, I can leverage the power
of overlays</em>4</a></sup>:</p>
(self: super: {
  vimPlugins =
    super.vimPlugins
    // {
      lspfuzzy = super.pkgs.vimUtils.buildVimPluginFrom2Nix rec {
        version = "latest";
        pname = "lspfuzzy";
        src = lspfuzzy;
      };
    };
})
</code></pre>
Applying this overlay to the Nix package set allows me to introduce a
new package called nix-env-fish into my locally available packages. It
works as a form of dependency injection, seamlessly integrating into my
configuration as if it were a part of Nixpkgs itself. The fact that it’s
added through an overlay doesn’t matter.</p>
Overlays are a bit like a gateway drug into authoring your own packages.
They let you quickly experiment with something locally, while still
using the infrastructure of Nixpkgs. Once your overlay is stable, it’s
easy to convert it into a standalone package and create a pull request.</p>
What didn’t work so well</h2>
Missing or Broken GUI Apps</h3>
At times, certain GUI apps on NixOS would simply display a black screen.
This issue appeared to be specific to Electron-based applications,
although I never found the motivation to debug these problems. On MacOS,
I faced another limitation as not all the GUI apps I wanted to install
were available. Some apps were either unavailable for the Darwin
platform or failed to launch. For example, Sublime Text
Editor</a>,
Obsidian</a>,
and Ledger Live
Desktop</a>
are examples of applications that are not accessible for Darwin systems,
as can be seen in each respective package’s “Platforms” list.</p>
Small Frustrations</h3>
A Nix and HM setup can come with its fair share of small
nuisances. I sometimes wish I were the type of person who kept a
meticulous journal with tags and all, as the issues I’ve listed here are
just the few that I can recall from memory.</p>
One of the trade-offs of having Nix manage your configuration files is
that it’s more or less no longer possible to edit them directly in place
(which is a somewhat obvious requirement for such a highly deterministic
build system). Even if you do make edits, they will eventually be
overwritten. If you tweak your Vim configuration file every five
minutes then this will really annoy you.</p>
On MacOS, launcher entries for applications may be missing. For example,
if I install the Alacritty terminal emulator through HM, I
can’t simply use “cmd+space” and type “Alacritty”, to launch it because
MacOS is unaware of its existence. Although HM creates the necessary
files, Finder does not index them. Here’s the GitHub
issue</a>.</p>
I was also missing some man pages at some point, but I think that was
fixable, and I think that this is the now closed GitHub
issue</a>.</p>
Integrating Nix and HM with Fish requires you to find some plugins
first. Initially I used
https://github.com/lilyball/nix-env.fish</a>
but later I switched to https://github.com/kidonng/nix.fish</a>.</p>
It took a few years until Visual Studio Code in Nix was in a state where
most things just work. It’s still a bit complicated to setup because
there are various options that all have some pros and cons. You can see
the documentation here</a>.
Also getting LiveShare to work on MacOS and NixOS requires you to be a
bit mindful about which plugins you need on which platform.</p>
Nix also demands more hard disk space, and the installer needs to create
a new volume. Fortunately, the Determinate Nix
Installer</a>
project nowadays provides a straightforward installation process on
MacOS.</p>
Added Complexity & Home Manager Abstractions</h3>
This was the reason I tried to live without Nix and HM for a while!
Nixpkgs is an enormous repository, and the Nix language itself adds to
the complexity. According to tokei</code>, the
nix-community/home-manager</code></a>
repository has over 50_000 lines of code! It can be a bit unsettling to
think about the sheer number of the layers of technology involved in
“just” installing a few applications and placing files in specific
locations.</p>
Of course, for the most part you don’t care how many lines of code go
into home-manager</code>. Similar to how I have no idea about the size of Vim
and Neovim, yet I rely on them daily.</p>
However, you’ll notice the complexity when something breaks. Fixing
issues in individual packages is usually manageable, but problems
stemming from the underlying infrastructure of Nixpkgs can be daunting.
Nix, being a functional, domain-specific language, is used with a
significant level of abstraction in Nixpkgs. It can be frustratingly
difficult to trace the origin of certain function arguments for example.</p>
I also have some doubts about the level of abstraction in HM.
For instance, the existence of options like
programs.neovim.enable</code></a>
in both HM and
NixOS</a>
is highly confusing. And it’s not just confusing for beginners; I still
find it confusing after 2 years. Are these options equivalent? How can I
spot the differences? Options like programs.alacritty.settings</code> allow
you to write Nix code that gets translated into a .yaml</code> file, which is
neat, I suppose. However, I personally prefer the less sophisticated
approach of directly writing .yaml</code> inline within my Nix script (or
write a separate .yaml</code> file and import it). Otherwise I need to
understand both the upstream configuration and the Nix equivalent of it.
Many packages also have an extraConfig</code> option for adding configuration
that doesn’t have a HM equivalent. This abstraction level
sometimes feels a bit off. I assume it creates a significant maintenance
burden, as upstream configuration options need to be reflected in Home
Manager options. Of course, you can ignore all this and manually create
your configuration files with xdg.configFile</code>.</p>
I’d probably find the level of abstraction of
nix-darwin</code></a> better, but I haven’t
had the time and energy to try it yet.</p>
At the end of the day I really don’t need the per-user installation of
packages and elaborate modules that HM gives me. I’d be perfectly
content with providing a list of packages to install system-wide and a
few basic primitives to generate configuration files in my home folder.
Having said all that, I’m still a happy HM user.</p>
Exploring Life Beyond Nix & Home Manager</h2>
I recently decided to give living without Nix and HM a try.
This decision was actually inspired by a brief conversation I had with
Gabriella Gonzalez</a> years ago. I had
noticed her using an incredibly simple setup during some Twitch streams,
so I reached out and asked about it. She kindly provided thoughtful
replies (thank you, Gabriella!) and, at least back then, seemed to be
using a mostly stock macOS setup. That got me thinking… if someone
like her can live without a billion fancy shell integrations and still
be a billion times more productive than me, maybe I could achieve the
same level of competence by emulating her approach?5</a></sup></p>
So I reverted to a simpler
configuration</a> where I relied
on MacPorts for installing my system-wide tools. I kept Nix around for
running nix develop</code> and accessing the project-specific environments I
mentioned earlier. However, I abandoned the fancy automation.</p>
So, how did it turn out?</p>
Well, typing nix develop ~/path/to/flake#project</code> a thousand times a
day isn’t exactly fun. And I dread the day I install a Neovim plugin
that requires a Rust toolchain. Instead of using a Vim/Neovim plugin
manager, I resorted to learning how packages work and ended up becoming
my own package manager with Git submodules. As for tmux and Fish
plugins, I simply stopped using tmux altogether. It’s a bit annoying
because Alacritty lacks splits and tabs. Going without Fish plugins
means living without a Git prompt in my shell, as the bundled Git
prompts are unusable in large repositories like nixpkgs</code>. Without
nix-direnv</code>, I no longer have the convenience of automatic sourcing of
environment variables. Every time I try to replay a database dump on my
local Postgres instance, I’m reminded of this when I encounter the error
message: AWS_S3_REGION is not set.</code></p>
Also, it’s quite funny when you run nix-store --gc</code> to reclaim some
space, only to realize that nix-direnv</code> was also preventing your
development environments from being cleaned up (check out this this
insightful
post</a>).</p>
But here’s the thing: Despite its flaws and complexity, I’ve become
incredibly accustomed to the convenience of running nix flake update</code>
and having almost everything on my system updated. I’ve grown accustomed
to typing cd</code> and witnessing my environment magically and instantly
switch.</p>
Resources</h2>

Determinate Systems Nix Installer</a></li>
kidonng/nix.fish</code></a></li>
My dotfiles</a></li>
Home Manager</a></li>
Very good blog post series about Nix</a></li>
Twitch videos by (among others) Gabriella Gonzalez</a></li>
Nix direnv</a>

Nix direnv project shells</a></li>
</ul>
</li>
</ul>

^{1</sup>
Sometimes you want to install a tool that’s not yet packaged up in
your system package manager of choice. In such cases it can be
tempting to just use something like npm</code> or cargo</code> to install
something globally in your system.</p>
</div>}^2</sup>I really like
PATH_add</code></a>
to automatically add ./node_modules/.bin/</code> to your $PATH</code> whenever
you are in a NodeJS project. That way it’s trivial to use the
project-local formatter and linter, and so on.</p>
</div>
^{3</sup>
You can manage your developer machine, build applications (kind of
like Bazel), deploy code to remote machines, define those machines,
and more.</p>
</div>
^{4</sup>
I’ve ellided some details. You also need to add this input to your Nix Flake, so it can keep track of which version you’ve installed. This is done like this:</p>
</div>
{
  lspfuzzy.url = "github:ojroques/nvim-lspfuzzy";
  lspfuzzy.flake = false;
}
</code></pre>
^{5</sup>
It didn’t work.</p>
</div>}}}


Building a Website With Haskell & Nix
2021-09-23T00:00:00+00:00
Introduction</h2>
I built a website so trivial that you might wonder why I even bother writing
about it. It consists of a static site which is, unfortunately, still built
with Gatsby, a Javascript framework. The content comes from a content
management system called Contentful. The most complex part is the members-only
area, which is the part I rebuilt and what this post is all about.</p>
The members area consists of a handful of routes:</p>

A news feed where administrators can post new messages. It’s plain text with the exception of URLs, which are replaced with proper anchor links. No images.</li>
User management area, where administrators can invite new members or edit and delete existing members. Here you can also filter by user groups and send emails to users you select through checkboxes. That’s the part where I used progressive enhancement. This route also hooks into email sending functionality for messaging someone who was invited to the site.</li>
People can create events, and others can then sign up for these events and also indicate how many guests they bring. Everyone can see who’s coming and who declined, how many guests there are in total. Events can also have attachements, such as PDF files, which are stored on the droplet itself.</li>
People can request a link that let’s them change their password.</li>
</ul>
The first version was hacked together with Firebase and some serverless functions,
because I didn’t want to maintain a proper backend. The UI was awkwardly fit
into the static site. It worked but it was ugly.</p>
And so one day I decided that it was time for a rewrite! The official reason is
that I considered it too risky to depend on Firebase for everything. Also the
local developer experience was a bit lacking at that time. The honest answer is
that, for quite a long time, I had been looking for a real-world project to
which I could apply all the fancy functional programming (FP) technologies I
picked up over the years. I was holding an FP-shaped hammer and I was just
looking for a suitable nail. This project seemed perfect, since it’s small,
has actual users and even a deadline, but since it’s a project for a family
member there’s no money on the line and I could do whatever I wanted in terms
of tech. And so the idea was born to rewrite the entire website with Haskell,
Nix and PureScript.</p>
In this post I want to share my experiences building this simple backend in
Haskell, Nix and PureScript. You don’t need to know any of these technologies
to follow the blog post. You might not understand every little detail, but I
hope that the overall message is still valuable. I’ll start the post with a
short summary, so you don’t have to read the entire article if you just want
the main conclusions.</p>
Here’s the source code</a>.</p>
Summary</h2>
Haskell</h3>
I’m neutral about using Haskell again in the future.</p>
On the one hand, algebraic data types and an excellent type system make it very
easy to translate my thoughts into code, since branching control flow, based on
a finite number of options, seems to make up the majority of my coding. It’s
kind of liberating to make sweeping changes across the code base, knowing that
the compiler will prevent a lot of mistakes. Haskell also has a surprisingly
large selection of web development frameworks and libraries. I didn’t have to
implement any mission critical functionality myself (encryption, database
communication, routing, and so on).</p>
On the other hand, one of the biggest time sinks was mere plumbing. Many
Haskell libraries revolve around specific Monads, created just for that
library. You don’t need to know what Monads are for this blog post, but the
gist is that you often need to write a little bit of glue code, so that
different Monads of different libraries can talk to each other. I’ll write more
about this later. But the end result was I spent way more time on glue code
than I thought. To make things worse, this glue code did not feel like
meaningful progress. It’s a Haskell solution for a Haskell problem after all.</p>
Haskell also has its fair share of historical baggage, which includes, but is
not limited to, an annoying module system, complicated tooling, and a rather
anemic standard library. The last point means you’ll have way too many lines of
code dedicated to imports.</p>
Nix</h3>
I would 100% use Nix again to provide a basic developer environment, which
includes compilers, formatters, database libraries, and so on. It is, in my
opinion, best in class in this area and entirely unrivalled. Think nix develop</code>.</p>
Whether or not I’d use it to build the actual project depends on the
programming language and how well supported it is in Nix. Haskell is the Nix
posterchild, Node on the other hand can be hit and miss.</p>
I would not use Nix again to deploy my code. It was fun playing around with
Systemd and SOPS, but the container ecosystem is just too big. Luckily you can
generate Docker containers with nix, which is something I’d like to explore in
the future. Running my NixOS image locally through QEMU was and still is,
frustrating, especially on MacOS. Additionally, my GitHub action workflow that
builds the application, runs end-to-end tests and deploys it, takes 25 minutes.
For a website of this size that is insane.</p>
SQLite</h3>
I love SQLite. It has quirks and historical baggage but at the end of the day
it’s a robust, battle tested and simple tool that I find to be a joy to use.
You can easily spin up an in-memory database for unit tests, it comes with a
surprising number of features, such as JSON tooling, and it’s just a file at
the end of day. This kind of simplicity is refreshing. Also shoutout to
Litestream</a> for database backups.</p>
10/10 would use again.</p>
PureScript</h3>
I’ve written 140 lines of PureScript (PS) code in this project, which is
nothing. It’s used for progressive enhancement of markup rendered on the
server, which means inserting DOM elements and interactivity into existing DOM
nodes. Unfortunately I couldn’t find any good PS libraries for this, so I had
to resort to writing very verbose and tedious code that looks like a one-to-one
translation from Javascript. All of the PS web frameworks and libraries I saw
want to own the markup they render, like React. So instead of surgically
injecting interactivity into existing DOM elements, you hand control over an
entire subtree of the DOM to PS. But that would have meant duplicating the
rendering (first render with Haskell on the server then with PS on the client)
of some routes in PS, which seemed like too much work.</p>
For this project PS was not a good choice, as it meant additional complexity
and an unnecessarily large bundle for little to no gain. I like the language
though. As such I just can’t say if I would use PS again in the future based on
this project alone.</p>
Deep Dive</h2>
Haskell</h3>
Haskell the language is really not complicated. But the ecosystem can be. I
frequently found myself spending too much time solving problems that only exist
in Haskell. These problems are usually the result of having to combine custom
Monads (which you can think of as small, domain specific languages) from
different libraries, something I alluded to already in the summary. For
example, I struggled quite a
bit</a>
with logging, which has only ever happened once before, in Clojure, where
logging is the final
frontier</a>.
But let me explain in a bit more detail why logging can be surprisingly tricky.</p>
Here’s a snippet from the README of
scotty</a>, a well known web
framework:</p>
main = scotty 3000 $
  get "/:word" $ do
    beam <- param "word"
    html $ mconcat ["<h1>Scotty, ", beam, " me up!</h1>"]
</code></pre>
It doesn’t matter if you know Haskell or not. I’d like to direct your attention
at param "word"</code>, or, applying the function param</code> to the string "word"</code>.
Maybe you can guess that this extracts the value of the route parameter we
defined in the preceding line, with "/:word"</code>. But isn’t it weird that we’re
not passing the HTTP request to the param</code> function? Where does it get
the request</em> paramter from, then? The answer is too complicated for this blog
post, but suffice it to say that there’s some magic going on behind the scenes.
And this magic is made possible by the Scotty Monad.</p>
Now, what if you want to also do some logging in your HTTP handlers? There’s a
really nice library called
katip</a>, and it also has
this very neat but also weird looking code:</p>
main :: IO ()
main = do
  -- ellided for brevity
  katipAddNamespace "additional_namespace" $ katipAddContext (sl "some_context" True) $ do
    $(logTM) WarningS "Now we're getting fancy"
  katipNoLogging $ do
    $(logTM) DebugS "You will never see this!"
</code></pre>
The katipAddNamespace</code> function does the same thing as any other structured,
hierarchical logging framework. Any log expressions in the indentend block of
code on the following lines will have this new namespace added to them. This
way the caller can add some additional context to the logger and the callee
doesn’t need to know or care about these things. As a proponent of structured,
hierarchical logging, I find this super neat. But, unlike in Go, where you’d do
something like</p>
newLogger := oldLogger.With("key", "value")
newLogger.Info("whatever")
</code></pre>
the newLogger</code> variable is nowhere to be seen. You might at first glance think
that we’re passing an anonymous function to katipAddNamespace</code> (the $ do</code>
part maybe?), but even if that were the case, we’re clearly not accepting any
arguments in that anonymous function. So what happens to the modified logger?
Well, the answer is, it’s complicated. Just like with Scotty, there’s a custom Monad
that takes care of all this plumbing for us behind the scenes.</p>
Of course the logger and its contexts and namespaces need to live somewhere. In
a typical setup you create a record that holds your application environment.
And in that environment you store the logger. You then need to teach Katip how
it can access and modify the logger in that environment. It’s essentially like
implementing an interface for a struct, if you’re a Go person.</p>
But then you also need to teach Scotty about this custom Monad. Instead of
working with
Web.Scotty</code></a>
you import
Web.Scotty.Trans</code></a>,
which states in its opening paragraph:</p>

The functions in this module allow an arbitrary monad to be embedded in
Scotty’s monad transformer stack in order that Scotty be combined with other
DSLs.</p>
</blockquote>
Which is exactly what I wanted to do. And at this point the whole house of
cards may or may not fall apart. Because custom Monads often come with
constraints. As in, if you want to run this custom Monad you need to make sure
that the context in which it runs supplies X, Y and Z. And then you start
wondering how those constraints will fit into the bigger picture of all the
other custom Monads you need to satisfy. For Scotty and Katip this ended up
being not too
crazy</a>,
but for another library – co-log</code> – I was unable to achieve my goal at all.
If this whole paragraph sounds a bit hand-wavy, please check out this Reddit
question</a>
and this StackOverflow
(SO)</a>
post.</p>
So yes, Haskell can be concise, expressive and type safe. But getting there can
be a pain, at least in the beginning. I do believe that over time this stops
being a problem, as you get more and more familiar with the Haskell type system
and the ecosystem.</p>
Aside from Monads, there was at least one other recurring topic that made me
scratch my head, and that’s how to deal with control flow and early return.</p>
Here’s what 90% of my Go code looks like:</p>
someValue, err := someFunc()
if err != nil {
  return errors.WithMessagef(err, "something went wrong with thing %s", id)
}
</code></pre>
Countless people have complained about the repetitiveness and boilerplate, but
I actually don’t mind it at all. The pattern is always the same, so it’s very
easy for me to follow the control flow inside a function. Additionally, it’s
trivial to return as early as possible. In fact, returning early is something
most people probably don’t even give a second thought to, since it’s just so
easy in imperative languages with statements.</p>
What does the above look like in Haskell then?</p>
case someFunc of
  Left err -> Left ([i|something went wrong with thing #{s}: #{err}|])
  Right -> -- keep going
</code></pre>
I suspect that many seasoned Haskellers will take fault with the above snippet
though, because it’s needlessly verbose. And I kind of agree, and I’ve spent
way too much time converting to and from various control flow patterns in this
project. とにかく, anyway, what’s going on here? someFunc</code> doesn’t return a
normal value and an error, it also doesn’t throw exceptions, rather it returns
a Result</code> type (called Either</code>). We then pattern match on the two possible
variations of this result type. Left</code> if there’s an error, Right</code> if
everything’s fine. Think Rust’s result</a>
type.</p>
But what if you have several function calls that all return different kinds of
results? Consider the following scenario:</p>

Unmarshal query parameters</li>
Check if user is authorized</li>
Get entity from database</li>
Perform some business logic with user and database entity</li>
</ul>
Pretty much all of these things can fail in some expected way (meaning
exceptions wouldn’t be appropriate). Here’s a snippet from an older commit that
shows what such code could look like.</p>
getTokenByValue dbConn token >>= \case
  Nothing -> return $ Left $ TokenNotFound token
  Just tok@Token {..} -> do
    ok <- hasUser dbConn tokenUserId
    if not ok
      then (return . Left $ UserForTokenNotFound tokenUserId)
      else do
        now <- Time.getCurrentTime
        if now >= tokenExpires
          then return $ Left $ TokenExpired tok
          else do undefined -- ...
</code></pre>
You can see where I’m going with this, right? The increasing indentation level
looks ugly. In all fairness though, this isn’t any more verbose than the
equivalent Go or Typescript code. I really wish that I had just stuck with
the ugly and verbose version</strong> instead of trying out various different
patterns. I believe that this code is simple, and readable and very easy to
understand. Refactoring this for vanity reasons is not something I’m proud of.</p>
There are some
tricks</a>
for dealing with the ugliness, but in my personal experience</strong> they often don’t work
in real world scenarios, where more complicated types are involved,
particularly anything with IO</code>. What does work is making liberal use of syntax
sugar, combinators and Monad transformers though.</p>
Here’s a snippet I copied verbatim from my code base, which shows what the
previous code looks like once you throw Monad transformers at the problem:</p>
token@Token {..} <- Token.get value >>= E.note' (NotFound value)
ok <- User.exists tokenUserId
E.unless ok $ E.throwError (NoUser tokenUserId)
now <- liftIO $ Time.getCurrentTime
E.when (now >= tokenExpires) (E.throwError $ Expired token)
return $ Valid token
</code></pre>
This is probably illegible to folks who are not familiar with Haskell, so here’s a hopefully human readable translation.</p>
Get some value and abort with a NotFound error if it's not there
Unless the user exists, abort with a NoUser error
Get the current time
When the current time is greater than the token expiration, abort
Return a valid token
</code></pre>
This version of the Haskell snippet doesn’t suffer from increasing indentation.
We also return early since behind the scenes Haskell still sort of pattern
matches on the various result types and it knows how it can short-circuit the
computation. For example, if ok</code> is false, this will immediately return the
NoUser</code> error, it will not run the remaining lines of that function. There’s
also very little to no Go-style line noise about error checking. I dare say
it’s expressive and concise. But it took me a lot of experimenting to get there
and also required writing some
utility</a>
functions for translating between various custom Monads, again. There are more things I
could complain about here1</a></sup>, but I hope that my main point here is clear:
translating a simple pattern, early return, to Haskell, without making
the code untolerably ugly, is not straight forward and it can be hard to find
this kind of advice in tutorials and books.</p>
That was a lot of negativity now, so let’s move on to something more positive:
algebraic data types (ADT) and pattern matching. Together, these two features
form the basis for how I model the world in code. Nothing beats the ease with
which this let’s me design branching control flow based on a finite number of
options. I currently write Go for a living and the lack of ADTs has caused more
than one logic bug and runtime panic. Generally, few languages give me the same
confidence as Haskell. It’s really hard to overstate the peace of mind that
comes from not having to worry about nil pointer exceptions and unhandled
cases. While I was writing this blog post I suddenly had an insight and removed
one field from a record that’s used everywhere in my application. In many
languages I would have dreaded this task but in Haskell I simply removed the
field and then followed the compiler errors. At the end I was pretty much 100%
certain that I didn’t break anything.</p>
There are a lot of other things I could write about here, but I think most of
the usual suspects have already been covered in great detail in other blog
posts, on Reddit or on Hackernews. This includes:</p>

Historical baggage like a string type no one really uses</li>
An anemic standard library resulting in lots and lots of imports</li>
No one likes records</li>
Slightly confusing build systems</li>
Modules that are at the same time tedious and not powerful</li>
Lazy evaluation can be tricky for performance and debugging</li>
The typical issues of working with a niche language</li>
…</li>
</ul>
But I don’t find any of these things particularly intersting and they’re also
not deal breakers for me.</p>
I wanted to write about the experience of using Haskell, not the nitty gritty
technical details. So what does it feel like, then? For the most part it’s just
like writing a backend in any other language. Define routes, parse query
parameters, do some authorization and authentication checks, fetch something
from the database, render a document. On a good day, Haskell makes me more
efficient and productive because I spend less time fixing bugs and less time
translating the problem to data structures, because ADTs and the type checker
are just that good. On a bad day, I still find myself deciphering runtime
issues, for example caused by a mismatch between database and Haskell types,
while also spending several evenings just solving Haskell issues.</p>
I’m neutral about using Haskell for future projects, because there are things
about the language I love and others that drive me crazy.</p>
Nix</h3>
According to tokei</code> this project has 1206 lines of Nix code, which fall into
three categories: developer environment, building parts of the application,
building and deploying the NixOS image that runs on a digital ocean droplet.</p>
I can’t think of a better way to provide all the tools necessary to work with a
project than Nix. Every new project I start uses Nix2</a></sup> to provide
instructions for how to create a shell that includes things like compiler,
formatter, database tools, terraform, and so on. In combination with
direnv.net</a> whenever I cd</code> into such a project my shell
environment is updated automatically. Doing this is as easy as adding the stuff
you need to a
list</a>,
and it usually just works (and keeps working).</p>
Building the various parts of this application with Nix was also not too much
work, but your mileage here depends on the languages and tools you use in your
project and how well they’re supported in Nix. For Haskell there’s a tool you
can use – cabal2nix</code> – which generates Nix code that contains instructions
for how to build your project, including its Haskell dependencies. You can
mostly just follow the documentation to get something up and running. But
Haskell is an outlier and many other languages don’t enjoy the same level of
support. For some of those languages there are tools that translate lock files
to Nix instructions, but they’re not without
issues</a>.
If no such tool exists you’re out of luck and building your application with
Nix will quickly become very, very challenging. Nix limits your ability to make
HTTP requests and interact with the file system. You therefore can’t just take
a command like yarn install</code>, which would download Javascript libraries, and
wrap it in some Nix code. Instead, Nix needs to download each of those
dependencies, and for that it needs at least a URL and a hash to verify the
contents. Good luck quickly whipping up a tool that does that.</p>
In this project I ran into little to no problems in that area though. What I
get in return for building everything with Nix is great caching of build
artefacts, and incredible reproducibility. I can clone the project, run a
single command, and it Just Works™. Additionally, even if it took me a few
hours to figure something out, it was usually a one-time cost.</p>
What did not spark a lot of joy was running my application with Nix, both
locally and remotely. Nix is already a niche technology, but deploying with Nix
is a niche within a niche. Documentation is lackluster and there’s a real
chance that you’ll be the first person to use a bunch of tools in a certain
combination and as such there’s no StackOverflow answer you can just copy from.
Here are some things that tripped me up in no particular order.</p>
You can’t build a NixOS image on a Darwin machine, so you need to run things
through Docker. It’s another layer of indirection and resource usage for no
gain. It’s generally frustrating if you develop on one machine for a while,
only to realize that everything’s broken when you to switch to another OS,
especially in 2021. I wouldn’t go so far as calling Darwin a second class
citizen in Nix, but you’ll have to put in extra effort here and there.</p>
If you run your application inside a NixOS image, how do you actually
(re-)start and monitor the application? I went with Systemd, which was another
piece of technology I needed to learn. Documentation is OK but outside of the
official manpages I found far fewer articles that talked about using Systemd
for backend services than I expected. It therefore took me longer than I wished
to figure out how to pass credentials to my application, share a database
directory between different Systemd services, which syntax goes where, and so
on. If I wanted to debug things on MacOS I needed to build the image through
Docker and then I also ended up running a QEMU VM through Docker. You’re
really on your own for a lot of these tasks and at some point I was tired of
figuring out one thing only be the stuck on the next task.</p>
The biggest time sink is just figuring out what arguments a Nix function takes,
which arguments are magically and silently passed through but can be customized
(and how), and what the function returns. My secret weapon here is GitHub code
search. Nothing beats seeing working code. I rarely benefit from reading the
official manual, since it doesn’t contain enough runnable examples. I do spend
a lot of time in issues, PRs and in source code.</p>
Consider Nix a slider that goes from local environment all the way to
production environment. The further right you go, the less likely it is that
you can just follow a cookbook. It’s therefore not a big surprise that I would:</p>

Always use Nix for local development</li>
Sometimes use it for building the application</li>
Probably not use it again for deployment</li>
</ul>
PureScript</h3>
This project was originally meant to be a super simple, single Haskell file
that implements a server that does not require any client side code. I really missed that mark.</p>
I added PureScript (PS) because there’s no HTML-only way of toggling a bunch of
checkboxes at once. The use case is to select a few (or all) users from a list
and then click a button to send an email with all selected users added as
recipients.</p>
The idea was to progressively enhance the server-side markup. On page load,
reveal the hidden “send email” button and inject a checkbox into every cell of
the users table. Do not replace the entire users table with markup rendered on
the client, because that would have meant reimplementing all of the markup in
PS. Or figuring out a way of sharing my Haskell template code with PS, but that
sounded like a lot of work. Unfortunately PS doesn’t seem to have any
libraries that help with that kind of task. Most of its ecosystem seems to
revolve around React-like libraries that want full control over a DOM node.</p>
So I ended up translating Javascript to PS almost word by word and as such the code looks like this:</p>
getCheckboxState :: Node -> Effect { email :: String, checked :: Boolean }
getCheckboxState checkbox = do
  checkboxAsEl <- maybe (throw "couldn't convert checkbox to element") pure $ Element.fromNode checkbox
  checkbox' <- maybe (throw "couldn't convert checkbox to input element") pure $ InputElement.fromNode checkbox
  isChecked <- InputElement.checked checkbox'
  email <- Element.getAttribute "data-email" checkboxAsEl >>= maybe (throw "no data-email attr") pure
  pure { email: email, checked: isChecked }
</code></pre>
This is the same as chaining document.querySelector</code> and getting and setting
DOM node attributes. Except that in PS I had to cast from one node type to
another a lot. It’s a bit puzzling and I’m still not sure if I overlooked
something really obvious, but take a look at
Web.DOM.Element</code></a>
and see for yourself. I think you’re not meant to write “low-level” code like
that. It’s probably better to wrap it in more expressive, high-level APIs.
Since I couldn’t find any of those in the form of ready-to-use libraries, and
didn’t want to abstract something I only did once, ugly code it is.</p>
I was pleasently surprised by the Nix support though, which seems to mostly be
the work of a single person,
https://github.com/justinwoo</a>.</p>
I wish I could say more about PS, because it’s actually a nice language, but it
just wasn’t a good fit for this project. I think the client side code is in the
kilobytes, whereas a vanilla JS implementation would have been much, much
leaner.</p>
Conclusion</h2>
Haskell is still my favorite programming language. Rust sounds like a modern
and more refined take on the concept of pair programming with a very capable
compiler, but whenever I tried it I quickly missed the ease of writing code in
a high level language. But as much as I enjoy using Haskell, I can’t ignore the
frustration it has caused me, and how much slower it often makes me during the
initial implementation of something.</p>
I still believe that Haskell, with a good enough understanding of its more
advanced language extensions, can be an incredibly productive language. The
more potential bugs you can rule out through the type system, the less
maintenance down the road. But getting there takes a long time.</p>
My goal for the next 12 months is to do another project of roughly the same
complexity level, but use tools that value simplicity above everything else.</p>
For example, instead of devising a complicated secret management solution, that
could end up being risky because you don’t understand how it works, I could
also just ssh</code> into the server and create a file with the secrets and call it
a day. Instead of using Nix to manage the complexity of a sprawling dependency
tree, what if my app barely had any dependencies?</p>

^{1</sup>
For example, I would strongly recommend to immediately catch potential
exceptions thrown by a function that has MonadThrow</code> (if I recall
correctly), because I’ve had surprising errors when the exception type was
not what I thought, because somewhere a function happened to have a
MonadThrow</code> in its signature but with the exception type hardcoded to
string. I’ve also created subtle bugs in code where I thought I was
short-circuiting but I was actually just returning IO Left</code> instead of
Left</code>. So this mtl</code>-style control flow is not without its pitfalls.</p>
</div>
^{2</sup>
More specifically a Nix Flake</p>
</div>}}


Ramda vs. Vanilla Javascript
2021-03-09T00:00:00+00:00
I’m a fan of functional programming (FP), especially when it’s coupled with a powerful type system as in Haskell. But my day job revolves around Go and Javascript and involves fairly little FP. Unfortuntately the only person I can blame for that is myself! Apparently I’m not doing enough internal lobbying to convince people of the merits of FP.</p>
If I’m honest, the reason I haven’t really pushed for FP is because I’m not convinced that it is indeed the better, default choice for teams that don’t have much FP expertise to begin with. Before I can convince anyone else, I’ll have to convince myself. Therefore I will start gathering examples of real life code that I can analyze through both the FP and the imperative lens. Hopefully over time I’ll come to a personal conclusion and it’ll have the nice side effect of giving me plenty of arguments for future discussions.</p>
Today’s example is a run of the mill data transformation. I need to transform a flat list of objects into nested objects, which will then be passed to the view layer, where each component will peel off one layer. In other words, I’m denormalizing data for maximum ease in the view layer. The snippets and code will include some place holders here and there and I won’t be mentioning the actual view layer and logic. The only thing that’s relevant here is getting the data from shape A to shape B.</p>
In the actual code the heading</code> is derived from first_name</code> and last_name</code> but in the benchmarks and examples I replaced it with a placeholder, since it’s irrelevant to this post.</p>
The code, including benchmarks, can be found on GitHub</a>!</p>
const input = [
  {
    section: "barsection",
    author: { first_name: "foo", last_name: "bar", id: "123" },
    content: "foo note",
  },
  {
    section: "barsection",
    author: { first_name: "bax", last_name: "bar", id: "111" },
    content: "bax note",
  },
  {
    section: "foosection",
    author: { first_name: "bert", last_name: "bar", id: "223" },
    content: "bert note",
  },
  {
    section: "foosection",
    author: { first_name: "bert", last_name: "bar", id: "223" },
    content: "another bert note",
  },
];

const output = {
  barsection: {
    len: 2,
    data: [
      { id: "111", heading: "bax'", contents: ["bax note"] },
      {
        id: "123",
        heading: "foos",
        contents: ["foo note"],
      },
    ],
  },
  foosection: {
    len: 2,
    data: [
      {
        id: "223",
        heading: "berts",
        contents: ["bert note", "another bert note"],
      },
    ],
  },
};
</code></pre>
If you focus on the output you’ll notice that we’re doing two aggregations here. We group the input by its section key and we also group all entries by the same author per section. The resulting data</code> value is an array so that the view layer can iterate over it without having to first call Object.values</code> on it.</p>
Vanilla</h2>
The vanilla solution is imperative to a fault. Personally I can’t glance at the code and immediately get an idea of what it’s doing on a high level. There’s too much plumbing going on, especially the checks and initializations to make sure we’re not accessing a key on an undefined object. Others will consider this an advantage though, because there’s absolutely no magic. Also note that the snippet doesn’t include any imports because there are none! One thing I like about it is that object initialization makes the shape of the data clearer. Additionally, it’s easy to see the execution model (e.g., number of iterations). Zero magic.</p>
const vanillaFn = (notes) => {
  const grouped = {};

  notes.forEach((note) => {
    const {
      author: { id: currentId },
      content,
      section,
    } = note;

    if (!grouped[section]) grouped[section] = { len: 0, data: {} };

    grouped[section].len += 1;

    if (!grouped[section].data[currentId])
      grouped[section].data[currentId] = {
        contents: [],
        heading: "",
        id: "",
      };

    const userNotes = grouped[section].data[currentId];
    userNotes.heading = "placeholder";
    userNotes.contents.push(content);
    userNotes.id = currentId;
  });

  Object.keys(grouped).forEach((sectionId) => {
    grouped[sectionId].data = Object.values(grouped[sectionId].data);
  });

  return grouped;
};
</code></pre>
Ramda</h2>
If you want to write functional JS in any significant capacity you’ll need a third party library. For many people lodash/fp</code> will be the first choice since lodash</code> (without the fp part) is probably already in use anyway.</p>
I would really encourage you to look into Ramda though, since it includes many functions that are simply missing from Lodash but which I’d consider an important part of the FP toolset and mindset. With Lodash you won’t get to experience the full power and conciseness of FP.</p>
Ramda’s rich API made me write two different versions of the code in question. I can’t help but notice that it’s quite typical for me to spend more time thinking about how to solve something when I’m using FP. Contrast this with Go where iterating means range</code> and that’s it. With FP you might spend the next hour musing about the pros and cons of various list comprehension and traversal interfaces.</p>
Version A</h3>
The first version makes heavy use of lenses to assign data at various levels of nesting. Other than that it’s essentially just a fold (reduce</code> in JS). The second operation just converts objects into arrays at the data</code> key (see the second code snippet below). I’d consider this version quite close to the vanilla implementation, since both operations in the call to pipe</code> map to a block of code in the vanilla implementation.</p>
const lensProp = require("ramda/src/lensProp");
const compose = require("ramda/src/compose");
const values = require("ramda/src/values");
const reduce = require("ramda/src/reduce");
const map = require("ramda/src/map");
const defaultTo = require("ramda/src/defaultTo");
const lensPath = require("ramda/src/lensPath");
const set = require("ramda/src/set");
const concat = require("ramda/src/concat");
const inc = require("ramda/src/inc");
const over = require("ramda/src/over");
const pipe = require("ramda/src/pipe");

const safeIncrement = pipe(defaultTo(0), inc);
const safeConcat = (xs) => pipe(defaultTo([]), concat([xs]));

const reducer = (object, { author: { id: currentId }, content, section }) => {
  const lenL = lensPath([section, "len"]);
  const userL = lensPath([section, "data", currentId]);
  const idL = compose(userL, lensProp("id"));
  const contentsL = compose(userL, lensProp("contents"));
  const headingL = compose(userL, lensProp("heading"));

  return pipe(
    over(lenL, safeIncrement),
    set(idL, currentId),
    over(contentsL, safeConcat(content)),
    set(headingL, "placeholder"),
  )(object);
};

const ramdaFn = pipe(reduce(reducer, {}), map(over(lensProp("data"), values)));
</code></pre>
const a = {
  dynamicKeyA: {
    dynamicKeyB: { foo: 1 },
    dynamicKeyC: { foo: 2 },
  },
};

const b = {
  dynamicKeyA: [{ foo: 1 }, { foo: 2 }],
};
</code></pre>
Version B</h3>
This version is quite different from the other two, and I’m finding it hard to correlate the individual operations to code in the vanilla implementation. The second part, replacing an object with an array of its values, is still the same. But the first part further differentiates between transforming a single input value and combining those transformed values. It doesn’t use lenses at all, and instead introduces the concept of transducers (hello Clojure!). In my opinion this is the cleanest looking version, because it elegantly skirts the issue of having to deal with operations on potentially uninstantiated, nested objects.</p>
const mergeDeepWithKey = require("ramda/src/mergeDeepWithKey");
const pipe = require("ramda/src/pipe");
const concat = require("ramda/src/concat");
const transduce = require("ramda/src/transduce");
const map = require("ramda/src/map");
const over = require("ramda/src/over");
const lensProp = require("ramda/src/lensProp");
const values = require("ramda/src/values");

const mapper = ({ author: { id: currentId }, content, section }) => ({
  [section]: {
    len: 1,
    data: {
      [currentId]: {
        heading: "placeholder",
        contents: [content],
        id: currentId,
      },
    },
  },
});

const merger = (key, left, right) => {
  switch (key) {
    case "len":
      return left + right;
    case "heading":
      return left;
    case "id":
      return left;
    case "contents":
      return concat(left, right);
  }
};

const ramdaFn = pipe(
  transduce(map(mapper), mergeDeepWithKey(merger), {}),
  map(over(lensProp("data"), values)),
);
</code></pre>
Discussion</h2>
We’ve looked at what the code is doing and how it’s doing that in three different ways. Now it’s time to wade into dangerous territory and attempt to compare FP and imperative programming in this tiny piece of code.</p>
I’ll analyze each snippet based on the following metrics:</p>

Performance</li>
Lines of Code & Dependencies</li>
Readability</li>
</ul>
If you think that I’m overlooking a really important metric please let me know at yuuki at protonmail dot com</code> or on whatever website I publish this post. Lastly, I’m not an academic. I’m sorry if the discussion of the individual metrics lacks rigor. Again, do let me know!</p>
Performance</h3>
This one we can measure! The repository I linked at the start of this post includes not just the different snippets but also a benchmark. The benchmark consists of a list of 3 input objects for two sections and two authors, which I repeat n</code> number of times. The resulting list of lists is flattened and then used as input to each code snippet. The test data isn’t very refined but it should be good enough for a relative performance comparison.</p>
n = 30
ramda x 1,067 ops/sec ±6.83% (69 runs sampled)
ramda merge deep x 6,301 ops/sec ±4.96% (77 runs sampled)
vanilla x 223,259 ops/sec ±5.09% (76 runs sampled)
</code></pre>
With an input list of length 30 the vanilla version is about 35 times faster
than the FP versions. Increasing it to 15000 makes the difference even more
drastic, with vanilla JS now 100 times faster. The difference between the two
FP versions is negligible.</p>
n = 15000
ramda x 2.19 ops/sec ±9.46% (10 runs sampled)
ramda merge deep x 7.70 ops/sec ±6.06% (24 runs sampled)
vanilla x 777 ops/sec ±3.42% (82 runs sampled)
</code></pre>
I also did a very basic</em> memory consumption test, by just commenting out some of the benchmark code and running the functions directly with $ command time -f '%M' node index.js</code>. Results:</p>
ramda            79756KB
ramda merge deep 79156KB
vanilla          32096KB
</code></pre>
Don’t take these values literally, they’re only useful for comparison purposes. Just like with speed, vanilla clearly comes out ahead.</p>
So what do we make of these results? I was expecting Ramda to be slow. I’d also suspect that if you can replicate some of the FP examples with lodash/fp</code> you’ll see a speed up. I’ve glanced at Ramda’s source and it implements the FP concepts with much more rigor than Lodash. As an end user you don’t care about that, but then again Lodash is missing a lot of important FP functions from its API.</p>
Personally performance is not my primary worry when writing client side Javascript. Clients shouldn’t have to deal with tens of thousands of objects. On the other hand Ramda is a lot</strong> slower. If you’re trying to maintain 60FPS you don’t have lot of wiggle room. I could very well imagine a rich client application that uses large enough lists for Ramda to become a problem.</p>
On the backend side you’re even more likely to work with sufficiently large data. Of course, always keep in mind that any network or file I/O might render these micro benchmarks meaningless. If your app is waiting 100ms for a request, it doesn’t matter much if your Ramda code is 5ms slower.</p>
My personal conclusion from this admittedly superficial look at performance is that while it doesn’t rule out Ramda, it means that Ramda will have to be really convincing in all other comparisons.</p>
Lines of Code & Dependencies</h3>
Why look at size (lines of code)? Users of concise languages will point out that there are studies that have shown that fewer LOC means fewer opportunities for bugs. Based on my own experience I agree with this statement but not without some nuance. One example in favor of reducing LOC is that in Go I constantly write exactly the same code for making a list of things unique, just with different types. At some point I’ll make a dumb mistake that goes unnoticed and lands in production. On the other hand really dense code can be incredibly hard to understand (looking at you Haskell). And lack of understanding is the perfect breeding ground for bugs. In the end I believe that a reasonable reduction in LOC is a good thing, but you shouldn’t become fanatic about it.</p>
$ wc -l vanilla.js ramda.js ramda_merge_deep.js
  31 vanilla.js
  34 ramda.js
  41 ramda_merge_deep.js
</code></pre>
There’s just not enough code to draw a meaningful conclusion here. The primary reason why the vanilla version is the shortest is that it doesn’t require almost a dozen imports. Without those things look a bit different:</p>
$ wc -l vanilla.js ramda.js ramda_merge_deep.js
  31 vanilla.js
  21 ramda.js
  32 ramda_merge_deep.js
</code></pre>
I think it’s safe to assume that across an entire code base Ramda will end up being significantly shorter. You might just eat the added bundle size that comes from importing the entire library instead of individual functions. Even if you keep the imports granular, they don’t add to the complexity of the code and in larger files their impact on LOC will diminish. I don’t have access to a code base that was written in both vanilla JS and Ramda so this will have to remain a reasonable assumption.</p>
Lastly, what is the impact of a library like Ramda on the overall bundle size? Depending on what you use as a point of reference, the 12kb</a> that Ramda adds to your bundle size might be completely irrelevant. Many sites ship megabytes of Javascript after all. But if you do care about size then those 12kb will have to pull their weight. Any dependency also adds a bit of maintenance work, although that’s less of an issue with general purpose utility libraries. It’s not like the API for map</code> changes every quarter.</p>
Readability</h3>
The final frontier. Few topics evoke such emotional responses as readability. It’s actually kind of funny that readability is often used as both an argument for and against Haskell. My biggest problem is that I don’t even know what I consider readable. I’m a web developer writing CRUD apps, as such ease of getting up to speed and developer velocity are really important for the code I write at work. Personally though I place a higher value on making the high level intent of code as clear as possible. But if you’re a systems developer then clarity of execution might be the most important aspects of readability. By this I mean things like how often does this allocate, when exactly is this future woken up, does this stream in chunks or not at all, and so on. And as I’m writing this I’m not so sure anymore if I’d really favor high level intent over clarity of execution model. Do I want to be known as a developer who wastes resources?</p>
It’s tempting to draw a line between high level and low level coding and define two different sets of readability rules. But about people who do both? Code that needs to be super fast but that’s dealing with really complicated logic?</p>
In an ideal environment I’d focus only on the logic and leave performance to the compiler. But such an environment does not exist. Therefore I will analyze the snippets both with regards to high level intent and clarity of execution model.</p>
The vanilla version makes it quite clear how many iterations will take place. There’s a forEach</code> at the start and another at the end. It also calls two Object</code> methods, which probably do some iterating as well. You’re also not left wondering if there’s any deep cloning going on (there isn’t).</p>
On the other hand it’s not apparent what this code is trying to accomplish. You really need to read it line by line to understand that we’re mostly gathering the content</code> fields of each input value, by section and author.</p>
The first Ramda version is more or less a translation of the imperative code but using lenses. Instead of forEach</code> it’s doing a reduce</code>, but other than that you need to check what each over</code> and set</code> is doing. To make matters worse, I had to create two wrappers so that I can use inc</code> and concat</code> without having to first check for undefined</code>.</p>
I’d argue that this snippet is the worst of the bunch. You lose the clarity of execution model from the vanilla version, without gaining much in terms of high level intent. Developers who are new to FP will be confused by lenses, especially once they see them being composed. Lastly, threading an object through a pipe of set</code> and over</code> will probably look super confusing. We normally don’t think of multiple assignments to object keys as multiple function calls.</p>
So what about Ramda version two? I absolutely love the way I can convert a single input value into a single output value without having to even think about nested objects paths. The shape of the output data is crystal clear as well!</p>
But then you stumble upon mergeDeepWithKey</code> and whatever clarity was gained by the mapping is immediately lost. The switch statement only specifies some keys. You really need to read the documentation to understand what happens with the rest. If you come from an FP background then you’ll probably make the connection between this and implementing semiring for your datastructure. But without that background knowledge it will seem pretty magical, and not in a good way. Additionally, you need to understand how the mapping and merging are used in the transduce</code> call. Clojurists won’t have any issues with that, but the average JS developer is unlikely to be familiar with transducers in general. Also I’ve seen it many times that JS developers are strangely averse to switch statements. I don’t have an explanation for this though.</p>
You could reduce the number of new concepts by separating the whole thing into a map</code> followed by a reduce</code> but then you’re back at square one. You’d need to write code that merges every key in both objects, potentially handling undefined</code> if your reduce</code> starts with an empty object.</p>
For me personally the third snippet is clearly the winner. It’s about as concise as it gets. It shows how to transform a single value. Then it shows how the results will be merged, while omitting keys that use the default merging strategy. But it also requires quite a lot of buy in from a developer who’s not familiar with FP.</p>
Conclusion</h2>
Ramda is much more concise, but also significantly slower than vanilla Javascript in this small example. The conciseness comes from using concepts such as lenses, transducers and higher order functions, which most developers are not familiar with. Careless use of functional programming concepts in a language without special support for that is, in my opinion, more likely to lead to unnecessary CPU usage and memory consumption. Of course you can write slow code in any paradigm, but if the most likely thing you’ll do is write a single loop and cram all your functionality in there, you’ll be more likely to experience a “pit of success” moment with regards to performance.</p>
Ultimately, I was hoping to answer the question: would FP be my default paradigm in my next JS project? I don’t know. I think that you’d need to invest a lot of time and energy into mentoring. You’d also need people curious enough to be mentored. I can guarantee you that some of the FP heavy code will come out needlessly complex and abstract in the beginning. This will lead to push back and endless discussions. I’m not sure if that’s really worth it? Maybe it’s better to use a language that was built with FP in mind rather than retrofitting it into JS.</p>


Exploring the Dhall configuration language
2020-04-28T00:00:00+00:00
Over the last weekend I looked into the Dhall configuration language</a>. I’m a fan of pure, functional programming in general and the works of Gabriel Gonzalez</a> specifically and so Dhall seemed like something I would like. I’m also not super happy with the way we handle kubernetes configuration files at work through helm, so I’m always looking for alternatives to that.</p>
In this post I’ll write down my first impressions of the language, which I got from creating type definitions and default values</a> for the alacritty terminal emulator</a>. In other words: I don’t have a lot of experience with Dhall yet and haven’t used it in a professional context. I’m not going to go through all the language features here, since the website does that much better. Instead, I’ll go over some things which stood out to me while exploring Dhall. After that I’ll show you an example of how Dhall can make working with configs safer and simpler for the end user</strong> before I get to the closing thoughts. Please also not that this is not an exhaustive introduction to the language. I’ll do my best to add context to all of the code snippets, but not all of them are beginner friendly if you’ve never worked with a pure, functional language before.</p>
All code snippets in this post use unicode symbols, so forall ()</code> becomes ∀ ()</code>. This is not required</strong> and purely for aesthetical reasons. Additionally, unless noted otherwise, all examples – both inline and code blocks – are runnable and don’t require any additional imports.</em></p>
Documentation & Getting Started</h2>
The first contact with a new language is usually the landing page and the languages’ documentation. In the case of Dhall both are excellent!1</a></sup> For a project so young it’s impressive how much documentation exists and how beginner friendly</strong> it is in general. Especially considering that Dhall has a fair share of overlap with the Haskell community, which is not exactly known for soft documentation. The best resource to get started with Dhall is, in my opinion, the language tour</a>, followed by the other tutorials on the page. Additionally, the Discourse</a> forum is active and very helpful.</p>
Dhall makes it super easy to just play around with little snippets of code. You can copy & paste this into your shell echo '{ key = "Value" }' | dhall</code> to see what it outputs1</a></sup>. Run it with dhall type</code> or dhall --annotate</code> instead of just dhall</code> to see more information about the expression. In general, exploring the different subcommands and flags of the dhall</code> binary is time well invested. You can also navigate to any file in your project and play around with its contents like so echo "(./file.dhall).someExport" | dhall</code>. And there’s also a dhall repl</code>! I love that kind of interactive development and it makes it super easy to get started with the language.</p>
Union Types</h2>
Having union types for configuration is generally amazing. It’s pretty common that a key only accepts a narrow range of values and union types let you represent those rules pretty much one to one in your code. The compiler will then verify the correctness of your configuration for you. That’s much more relaxed than having to do these checks manually.2</a></sup> This saves you time and energy since you don’t need to wait for your configuration to be deployed, and rejected, only to discover you made a typo somewhere.</p>
let Action = < Copy | Paste >

let Config = { action : Action }

-- Below doesn't compile since DangerousThing is not an allowed value!
-- let myConfig : Config = { action = Action.DangerousThing }

-- This is allowed!
let myConfig : Config = { action = Action.Paste }

in myConfig
</code></pre>
What’s slightly annoying about union types in Dhall is that there’s no way of extending them, or making one union type the subtype of another. Consider the following:</p>
let Type1 = < A | B | C >
let Type2 = < A | B | C | D >
</code></pre>
It’s clear that Type1</code> is “part of” Type2</code>, meaning I should be able to do Type2.A</code> and use that in any function accepting a Type1</code>. But that’s not (yet) a thing in Dhall, even though there’s an issue for it</a>.</p>
There are various ways of dealing with this though. In an early version of dhall-alacritty</code>, the base type would be imported into the more extensive types:</p>
let Base = < A | B >

let Extended = < Base : Base | C >
</code></pre>
This works but requires additional work if there are several variations of Extended</code> but you want to implement some shared functionality in the module where Base</code> lives, which has to work on these variations. Imagine a record, which can hold variations of Extended</code>, but the record as a whole mostly consists of shared keys and functionality. The below snippet shows one way of handling this situation in Dhall through the use of polymorphism. The snippet is big but it’s not actually that complicated! It really boils down to passing your extended union type and a function to handle that extension</strong> to the code implementing the shared functionality. In other words: the shared record and the functions operating on that record know almost everything about said record, except for one key and how to work with it. So you’re supplying them with this last piece of information to make them whole.</p>
The output of the code below is { extraKey = "C", sharedKey = "5", sharedKey2 = "10" }</code> by the way.</p>
(Also if you’re wondering what merge</code> does, it’s how you pattern match on union types, since there’s no actual pattern matching in the style of Haskell)</em></p>
let Prelude = https://prelude.dhall-lang.org/package.dhall

let Base = < A | B >

let showBase = λ(b : Base) → merge { A = "A", B = "B" } b

let Extended = < Base : Base | C >

let showExtended =
	  λ(e : Extended)
	  → merge { C = "C", Base = λ(b : Base) → showBase b } e

let Rec
	: ∀(extraType : Type) → Type
	=   λ(extraType : Type)
	  → { sharedKey : Natural
		, sharedKey2 : Natural
		, extraKey : extraType }

let handleRec
	:   ∀(extraType : Type)
	  → ∀(handleExtraType : extraType → Text)
	  → ∀(rec : Rec extraType)
	  → { sharedKey : Text, sharedKey2 : Text, extraKey : Text }
	=   λ(extraType : Type)
	  → λ(handleExtraType : extraType → Text)
	  → λ(rec : Rec extraType)
	  → { sharedKey = Prelude.Natural.show rec.sharedKey
		, sharedKey2 = Prelude.Natural.show rec.sharedKey2
		, extraKey = handleExtraType rec.extraKey
		}

in  handleRec
	  Extended
	  showExtended
	  { sharedKey = 5, sharedKey2 = 10, extraKey = Extended.C }

</code></pre>
It’s actually quite amazing that you can do this in your configuration language. Doesn’t mean you have to, of course, and I doubt that it will come up very often.</p>
Polymorphism</h2>
The above paragraph brings me to my next point: polymorphism (hope I’m using the word correctly) in Dhall is fairly straight forward. In a type signature, you can use the forall</code> keyword (turned into a pretty unicode symbol by dhall format</code>) to create a type variable and then refer to that variable (and type) in the rest of the signature. Like so (example taken straight from my toy project)</p>
Not a runnable example. I ellided the function body to focus only on the type signature</em></p>
let showBindings
	:   ∀(action : Type)
	  → ∀(show : action → Text)
	  → List (KeybindingIn action)
	  → List Keybinding
</code></pre>
Remember that part about extending union types from the paragraph above? That’s exactly what’s going on here. KeybindingIn</code> is used in 3 different contexts and each context needs to first decide which action</code> type to use with KeybindingIn</code> and how that action</code> type can be turned into a string. Armed with that knowledge showBindings</code> can do all the heavy lifting.</p>
This is quite powerful but can also impair readability of course. I dare say that most programmers would probably run away sreaming if you told them that their configuration language</em> requires them to understand snippets like the one above. Especially when their day job is something like Javascript or Go (disclaimer: that’s my day job). In reality you probably won’t need polymorphism in most cases. In the case of the union type you could just copy & paste the common constructors. It’s a bit of a maintenance burden, but I’d rather do that and gain some type safety, than throw out the Dhall baby with the polymorphism bath water. Just because powerful features exist, doesn’t mean you need to use them.</p>
Records</h2>
There are several helper functions and operators, which make working with records really enjoyable and straightforward in Dhall (that’s quite the feat and the same can’t be said about records in Haskell). You can shallow merge two records with the //</code> operator. As the word shallow implies, this will completely override nested records:</p>
{ a = { b = 1, c = 2 } } ⫽ { a.b = 1 }
</code></pre>
Returns { a.b = 1 }</code>, thereby losing the c = 2</code> part. In practice this is less of a danger than it seems, since you’d probably have a type signature making sure that c</code> is always present. There’s also a recursive merge operator, /\</code> but it does not allow for collisions. You can merge but you can’t override. So how do you override something that’s nested a few levels deep?</p>
{ a = { b = 1, c = 2 } } with a.c = 3
</code></pre>
I can’t overstate how amazing this is</strong>. You’d think that a statically typed, pure functional language would be more tedious at working with records than a dynamic language. Also note the use of the dot operator</strong> here, which is easy to read and write and makes working with nested records much more ergonomic.</p>
Records can also have schemas, which is just a record with two keys, one for Type</code> and one for default</code>.</p>
let rec = { Type = { b : Natural, c : Natural }, default = { b = 0, c = 0 } }

in  rec::{=}
</code></pre>
The ::{=}</code> creates the record with its default values and ::{ c = 5}</code> let’s you shallow override the defaults. It’s nice syntax sugar for creating a record with default values although you can achieve the same thing with the existing operators. I’m a fan of orthogonal features and so I’m not a big fan of this one (even though I’ve used it extensively in my own little toy project). Also this only works for records. { Type = Optional Natural, default = None }</code> does not compile. This means you’ll likely end up mixing defaults with syntax sugar and defaults without anyway.</p>
There’s also a feature called projections</a>. It’s a bit like destructuring in languages like JS or Clojure.</p>
let record = { a = 1, b = 2}

in record.{a}
</code></pre>
In the first version of this post I complained about not understanding when and how to use this, but the Discourse</a> community helped me with that. Still, I don’t find this feature as useful as it could be, since it’s a bit limited where you can actually use this. Or in other words, I think I was expecting this to be the equivalent of destructuring in JS and Clojure, which it’s not:</p>
Does not compile since that’s not how you use projections</em></p>
let record = { a = 1, b = 2}

in \(rec.{a} : Record) -> a
</code></pre>
Dhall doesn’t currently have actual destructuring for things like function arguments. Check out some of the discussions for such a feature at GitHub</a>.</p>
Also note that if you need to generate a record dynamically you can actually do so</a> without having to jump through any hoops, which is pretty neat. The reason I’m not emphasizing this more is that it didn’t really come up when I was working with Dhall for dhall-alacritty</code>, but it is something many people will want. It’s this attention to real world use cases</strong> that really makes Dhall shine.</p>
Ergonomics & Tooling</h2>
I’m using neovim</code> and the dhall language server kinda sorta works some of the time. It doesn’t have a lot of features, so it’s mostly about error checking and completion. In Visual Studio Code the experience is a bit better, but the features remain the same. The dhall</code> binary and its subcommands is how I explore my code and ask the compiler questions about it. To me things like dhall --annotate</code> or dhall type</code> kind of achieve the same thing as typed holes in Haskell.</p>
I’ve already mentioned a couple of ways for interactively working with Dhall code throughout this post (echo '{}' | dhall</code>, dhall --file ./file.dhall</code>), but there’s also dhall repl</code> which should probably be your first choice for playing around with Dhall. It has a :help</code> command so there’s no need for me to go through its commands.</p>
One thing I find a bit strange is that currently dhall format</code> removes comments. There’s an issue for that too</a> with some workarounds but it’s really annoying. Imagine introducing Dhall at work and then not easily being able to leave helpful comments for your colleagues who may be new not only to Dhall but also functional programming in general.</p>
Error messages in Dhall are generally really good but can be a bit hit and miss in certain cases. For smaller types (meaning here not too many keys, not too nested) they show you exactly what’s wrong and are easy to interpret. In some cases they even includes helpful hints for common syntax mistakes, which reminded me a bit of Elm. For a project this young I’d say errors are mostly a shining beacon of excellence. It’s only when working with huge records that the errors become more and more cryptic, since the size of the diff (expected vs actual) to some degree mirrors the size of the record. Meaning your terminal will suddenly be filled with a wall of text and lots of ...</code> to hide some lines deemed irrelevant for the error. All in all errors are mostly really good though. For example, here’s one of those errors that also displays some helpful hints:</p>
r : { Type : Type, default : Optional Natural }

Error: You can only override records

Explanation: You can override records using the ❰⫽❱ operator, like this:


	┌───────────────────────────────────────────┐
	│ { foo = 1, bar = "ABC" } ⫽ { baz = True } │
	└───────────────────────────────────────────┘


	┌─────────────────────────────────────────────┐
	│ λ(r : { foo : Bool }) → r ⫽ { bar = "ABC" } │
	└─────────────────────────────────────────────┘


... but you cannot override values that are not records.

For example, the following expressions are not valid:


	┌──────────────────────────────┐
	│ { foo = 1, bar = "ABC" } ⫽ 1 │
	└──────────────────────────────┘
								 ⇧
								 Invalid: Not a record


	┌───────────────────────────────────────────┐
	│ { foo = 1, bar = "ABC" } ⫽ { baz : Bool } │
	└───────────────────────────────────────────┘
								 ⇧
								 Invalid: This is a record type and not a record


	┌───────────────────────────────────────────┐
	│ { foo = 1, bar = "ABC" } ⫽ < baz : Bool > │
	└───────────────────────────────────────────┘
								 ⇧
								 Invalid: This is a union type and not a record


────────────────────────────────────────────────────────────────────────────────

You supplied this expression as one of the arguments:

↳ None Natural

... which is not a record, but is actually a:

↳ Optional Natural

────────────────────────────────────────────────────────────────────────────────

1│                                                                r::{=}
</code></pre>
What Dhall Can Do For You</h2>
As promised, here’s an example of how I used Dhall to model the rules for key mappings in alacritty. I’d say that this is not beginner level Dhall code anymore, especially if you’re not familiar with some of the functional programming concepts in use. But I hope that it still illustrates just how much you can achieve in terms of type safety without having to do mental gymnastics and importing various PhD theses, thanks to Dhall’s focus on real world use cases.</p>
The alacritty configuration file let’s you map keys to actions. This is fairly involved so I’ll use a slightly simplified model so the code snippets don’t blow up in size. Here’s a mapping that binds the “A” key, when “Alt” is held down as well, to the “Copy” action.</p>
key_bindings:
  - key: A
	mods: Alt
	action: Copy
</code></pre>
Modifiers and actions aren’t just random strings though. There’s a list of possible values for both, so let’s model those rules in Dhall:</p>
let Modifier = < Alt | Control | Command >

let Action = < Copy | Paste >

let Binding = { mods : Optional Modifier, key : Text, action : Action }

let myBindings : List Binding =
		[ { mods = Some Modifier.Alt, key = "A", action = Action.Copy } ]

in  myBindings
</code></pre>
Executing this file with dhall-to-yaml --file ./file.dhall</code> gives me exactly the keybinding from above (without the actual key_bindings</code> key). Yay for more type safety! It also improves the developer experience since you now get reliable autocompletion for allowed values! There’s just one issue: this code doesn’t support multiple modifiers. It should be possible to eventually render a .yml</code> file with an entry like this:</p>
key_bindings:
  - key: A
	mods: Alt|Shift
	action: Copy
</code></pre>
The mods</code> key will now have to hold a list of modifiers and then we’ll also need to create a function that turns List Modifier</code> into Text</code> by joining the modifiers with a |</code>.3</a></sup></p>
let Prelude = https://prelude.dhall-lang.org/package.dhall

let Modifier =
	-- List the possible modifiers we have
	< Alt | Control | Command >

let renderModifier
	: Modifier → Text
	=
	  -- Write a function that converts a single modifier to its text representation
	  λ(m : Modifier)
	  → merge { Alt = "Alt", Control = "Control", Command = "Command" } m

let convertModifiers
	: List Modifier → Text
	=
	  -- Convert a list of modifiers to a string where the modifiers
	  -- are joined with a pipe
	  λ(m : List Modifier)
	  → let mapped = Prelude.List.map Modifier Text renderModifier m

		let joined = Prelude.Text.concatSep "|" a

		in  joined

let Action = < Copy | Paste >

let Binding =
  -- Binding now let's you specify multiple modifiers for a given keybinding
  { mods : Optional (List Modifier), key : Text, action : Action }

let BindingRendered
	: Type
	= { mods : Optional Text, key : Text, action : Action }

let renderBinding
	: Binding → BindingRendered
	=
	  -- Use the various functions from above to convert just a
	  -- single key in Binding and then use that converted key
	  -- and value to create the rendered version of Binding
	  λ(b : Binding)
	  → let mods =
			  Prelude.Optional.map (List Modifier) Text convertModifiers b.mods

		in  { mods } // b.{action,key}

let myBindings
	: List Binding
	= [ { mods = Some [ Modifier.Alt, Modifier.Control ]
		, key = "A"
		, action = Action.Copy
		}
	  ]

in  Prelude.List.map Binding BindingRendered renderBinding myBindings
</code></pre>
😱! Let’s go through this (I also annoted the file with comments!). The definition of Modifier</code> is the same, but I’ve added a function for converting a value of that type to text (renderModifier</code>). It uses the aforementioned merge</code> function which let’s you pattern match on different constructors of that union type. Here I’m simply mapping each union type value to a string (Text</code>) of the same name.</p>
The next function is a bit more involved, but the type signature is an excellent first thing to look at. convertModifiers</code> takes a list of modifiers and outputs a single string. What it does is convert each modifier to a text then combine them all with | in between</strong>. I could also write this in a more compact manner:</p>
You can run this snippet by replacing convertModifiers</code> in the snippet above</em></p>
let convertModifiers
	: List Modifier → Text
	= Prelude.Function.compose
		(List Modifier)
		(List Text)
		Text
		(Prelude.List.map Modifier Text renderModifier)
		(Prelude.Text.concatSep "|")
</code></pre>
The version using compose</code> simply emphasises the concept of composition a bit more: the list of modifiers flows through a pipeline of “list of modifiers” -> “list of texts” -> “text”, which is just a chain of two function calls. That chain can also be expressed as the composition of these two functions. I’d always go with the more verbose and explicit version first, unless I’m working in a team where I know everyone is comfortable with composition.</p>
Back to the bigger snippet. Binding</code> now reflects the changed business logic</strong> through List Modifier</code>, and I’ve added a rendered version of binding. Notice how I need to repeat key</code> and action</code> though – there’s no merge & override type operator.</p>
Lastly I’m putting all of this together with renderBinding</code>. There’s something really amazing going in there, which you may not know from other languages. Notice how inside renderBinding</code> I’m never checking if a value is Some or None</strong>, even though both Binding</code> and BindingRendered</code> contain optional types. All of this plumbing is taken care of by Prelude.Optional.map</code>. If you’re coming from Elm, Purescript or Haskell this will be familiar. If not, it’s like syntax sugar for if (isSome) then doThis else doThat</code>.</p>
So what’s the output of the above snippets?</p>
- action: Copy
  key: A
  mods: "Alt|Control"
</code></pre>
Is it worth it? I think so. It’s a lot of code but it’s also a lot of safety. Everyone using that code to generate their mappings will benefit from the initial effort. To me code like this has a great multiplicative impact on others. Also some of the verbosity of Dhall comes from the fact that polymorphic functions require you to explicitly specify types. In Haskell you could simply do map someFunc someOptional</code> and the compiler would infer the necessary types for you. In Dhall you need to explicitly state the input and output types of the map</code> call.</p>
Closing Thoughts</h2>
Would I use Dhall at work? Absolutely. As I mentioned, I’m a fan of functional programming (FP). Unfortunately there’s only a single developer I’ve met in real life (outside of FP meetups) who’s interested enough in FP so that she started learning Elm on her own. There are many reasons why people aren’t as excited about FP as I’d wish for, but one of them is simply that there are lots of alternatives. If you’re doing just fine with JS, why look into something else? And that’s precisely why I think Dhall has a lot of potential to introduce a bigger audience to FP: there aren’t that many alternatives in this problem space.</p>
Our kubernetes and helm configuration at work comes down to a mix of .yaml</code> files with Go templating directives in strings, held together by conventions. There’s nothing which verifies the correctness of the logic in your templates. Stuff like {{- with .Values.labels }}</code> and</p>
{{- default .Chart.Name .Values.nameOverride |
  replace "." "-" |
  lower |
  trunc 63 |
  trimSuffix "-" -}}
</code></pre>
is a bug waiting to happen. It’s also super exclusive. Imagine a junior developer trying to work with these files, without being able to verify that what they’re doing makes any sense. Without understanding where .Values</code> comes from, what’s inside, what you can do with it. There’s often no explicit connection between the variable in one file and its definition in another (I say junior developer not to belittle them – I also often don’t know exactly what values I’m working with in these files)</em>.</p>
Dhall isn’t the only contender in this area but in my opinion it’s the one best equipped to solve these issues. Dhall let’s you ramp up the type safety gradually, it gives you some</em> programming features without letting you do whatever you want by limiting the language features it provides. You can improve the type safety and</strong> ergonomics of your configuration files with minimum effort and without any fancy types. You can also invest a lot of time and effort into making something critical as type safe as possible.</p>
Dhall has a great and unique vision of how to apply a subset of FP to configuration files and I think it has great potential. Also you can easily learn the basiscs in a couple of hours so I’d encourage you to give it a try. The only thing you need to get started is the Dhall compiler and a terminal.</p>
^{1</sup>
In bash</code> you can also use a HERE string (<<<</code>) but I use fish</code> so I’m using echo</code> and pipe.</p>
</div>
^{3</sup>
At least that’s one way of doing it. You could also change the type of mods</code> to Text</code> and just expect people to apply some renderListOfModifiers</code> function to their List Modifier</code> before</strong> creating the Binding</code> record.</p>
</div>
^{2</sup>
You are only human – Genji</p>
</div>}}}


Understanding unliftio
2019-05-30T00:00:00+00:00
UPDATE 2025: Here’s an updated gist</a> that uses a Nix shebang for a single, executable .hs</code> file that you can run</strong></p>
In this post I will go over the source code for unliftio-core</a>, more specifically the MonadUnliftIO</code> typeclass, found in Control.Monad.IO.Unlift</code>. The typeclass is used to power the actual unliftio</a> library, which is the one you’d use in your applications.</p>
Let’s start with an example (borrowed from the official docs) showing what unliftio</code> is good for.</p>
UnliftIO in a Nutshell</h2>
You have the following function – which gives you an IO ()</code> – but you actually need ReaderT env IO ()</code>.</p>
foo :: String -> IO ()
foo = print
</code></pre>
The most straight forward and idiomatic way to translate from IO a</code> to m a</code> is liftIO</code>.</p>
bar :: String -> ReaderT env IO ()
bar = liftIO . foo
</code></pre>
But now you have a function where IO</code> occurs in negative position1</a></sup> (see this post on co- and contravariance</a>), as is often the case with handlers passed in as arguments.</p>
foo :: (String ->  IO ()) -> IO ()
                -- ^^^^^ negative position
foo func = func "test"
</code></pre>
liftIO</code> won’t help us, but unliftIO</code> can solve this problem.</p>
foo :: (String ->  IO ()) -> IO ()
                -- ^^^^^ negative position
foo func = func "test"

foo2 :: (String -> ReaderT String IO ())
   -> ReaderT String IO ()
foo2 func =
  askUnliftIO >>=
  \u -> liftIO $ foo (unliftIO u . func)
              -- ^^^ note that we're still calling
              -- the original `foo`. We didn't have to
              -- reimplement anything

-- Or alternatively and more concisely
foo2' func =
  withRunInIO $ \runInIO -> foo (runInIO . func)
</code></pre>
Both versions of foo2</code> work. unliftIO</code> translates from m a</code> to IO a</code> so that we can utilize our ReaderT</code>, but inside a function expecting plain IO</code>. In that sense the name “unlift” is quite fitting since it’s the opposite of lifting: it allows us to preserve the monadic context (for example the environment from a reader) but run it in IO</code>.</p>
What made me scratch my head a little was where runInIO</code> and u</code> came from, and how they work. Exploring those two questions is therefore the topic of this post.</p>
The MonadUnliftIO</code> typeclass has two methods: askUnliftIO</code> and withRunInIO</code>, either of which suffices for a minimal implementation. We’ll go over both, so let’s start with askUnliftIO</code>.</p>
askUnliftIO</h2>
This method has the function signature m (UnliftIO m)</code> and returns a newtype wrapper (shown below), which, according to the documentation, exists because</p>

We need to new datatype (instead of simply using a forall) due to lack of support in GHC for impredicative types.</p>
</blockquote>
newtype UnliftIO m = UnliftIO
  { unliftIO :: forall a. m a -> IO a
  }
</code></pre>
If you’re as baffled by impredicative types</a> as I was, feel free to peruse the link. Or don’t, because it’s not necessary to understand how unliftio</code> works. You can actually ignore the newtype wrapper entirely, for the purposes of understanding how the library works, since it’s purely an implementation detail.</p>
To make it easier to follow along, I created a gist</a> with a Haskell script that you can simply save somewhere and then load into ghci</code>. Note that I’m not importing unliftio</code>, instead I simply copy & pasted the minimal, relevant bits from the source code.</p>
We have two instances for MonadUnliftIO</code> in this file, one for ReaderT</code> and one for IO</code>. The latter isn’t very interesting, since it’s more or less just the identity function. We’ll therefore focus on the former:</p>
instance MonadUnliftIO m =>
  MonadUnliftIO (ReaderT r m) where
  askUnliftIO =
  ReaderT $ \env ->
    askUnliftIO >>= \newtypeU ->
    let unlift = unliftIO newtypeU
     in liftIO $ return
       (UnliftIO (unlift . flip runReaderT env))
</code></pre>
The function signature of askUnliftIO</code> specialized to reader is ReaderT r m (UnliftIO (ReaderT r m))</code>. Since we need to stay in the ReaderT</code> monad, the function body starts by creating a new ReaderT</code>. More precisely, and also how the docs formulate it, we’re preserving the reader’s monadic context</em>.</p>
In the function passed to the ReaderT</code> constructor, we use askUnliftIO</code> yet again, but this time it’s the instance of the monad inside our reader (the m</code> in ReaderT r m</code>). That’s why there is a type class constraint, mandating that m</code> also implements MonadUnliftIO</code>. This kind of unwrapping our monad layers by invoking the instance for the next layer is a pretty important concept which can be seen in many libraries.</p>
Since we’re using monadic bind >>=</code>, the inner most function will get the UnliftIO m</code> part from m (UnliftIO m)</code>, which is called newTypeU</code> in the snippet. Once unwrapped, this gives us the actual function to unlift</strong> something from m a</code> to IO a</code> – here called unlift</code>.</p>
Quick reminder about the syntax here: the UnliftIO</code> newtype is a record with a single field called unliftIO</code>. To get that field’s content from the record, we apply the unliftIO</code> function to the record – just like with any other record in Haskell!</em></p>
That unlift function is used in the final line of the snippet where we first run our reader (remember that we want to preserve the monadic context) and then pass the result to our unlift</code> function. In our case with ReaderT env IO</code>, that unlift</code> is just the identity function</strong>, since that’s how the unlift instance is defined for IO</code>.</p>
instance MonadUnliftIO IO where
  askUnliftIO = return (UnliftIO id)
</code></pre>
In other words, composing the two functions creates a new function which takes a monad (ReaderT</code>) and outputs an IO</code>, which is exactly the signature (m a -> IO a</code> ) we need. We then package that composed function up in UnliftIO</code>, and return a lifted version of it. 2</a></sup></p>
TL;DR</strong>: What this achieves is that calling askUnliftIO</code> on a reader</p>

calls askUnliftIO</code> of the monad inside the reader to recursively unwrap the monad layers until we get to IO</li>
runs the reader (its monadic context)</li>
repackages it all in a reader and in an UnliftIO</code>, giving us an unlift function</li>
</ol>
One thing that helps tremendously is to just add type annotations everywhere and let GHC check if our assumptions are correct. If you’re ever stuck and can’t figure out the type signature of something, just replace it with a wildcard3</a></sup> and see what GHC suggests. Here’s the above code snippet but with types sprinkled all over it. Note that this snippet requires the {-# LANGUAGE InstanceSigs #-}</code> pragma</em>!</p>
instance MonadUnliftIO m => MonadUnliftIO (ReaderT r m) where
  askUnliftIO :: ReaderT r m (UnliftIO (ReaderT r m))
  askUnliftIO = ReaderT f
  where
    f env =
    (askUnliftIO :: m (UnliftIO m)) >>= \(u :: UnliftIO m) ->
      let unlift = (unliftIO u :: m a -> IO a)
        unlift' =
        (unlift . (flip runReaderT env :: ReaderT r m a1 -> m a1))
        returned =
        return (UnliftIO unlift') :: IO (UnliftIO (ReaderT r m))
       in liftIO returned :: m (UnliftIO (ReaderT r m))
</code></pre>
withRunInIO</h2>
The withRunInIO</code> function removes a bit of the plumbing that askUnliftIO</code> requires and let’s you write code that is even more concise than askUnliftIO</code> (as can be seen from the example at the end of the first chapter).</p>
Its function signature is</p>
((forall a. m a -> IO a) -> IO b) -> m b
</code></pre>
but for simplicity’s sake I won’t include the forall</code> part in future references to the signature. As you can see, it takes only a single argument, which is a callback.</p>
Just as with askUnliftIO</code>, the IO</code> instance is pretty boring, so we’ll jump straight to our familiar ReaderT</code>.</p>
The basic layout is quite similar as for askUnliftIO</code>: We’re returning a ReaderT</code>, and inside the function passed to the constructor we’re calling withRunInIO</code>. This uses the withRunInIO</code> instance from whatever monad we’re using in the reader. Hence the type class constraint for the monad used in the reader. But there is one important difference: withRunInIO</code> takes an argument, which is called inner</code> in the code snippets.</p>
withRunInIO inner =
  ReaderT $ \env ->
  withRunInIO $ \runInIO ->
    inner (runInIO . flip runReaderT env)

foo2' func = withRunInIO $
  \runInIO -> foo (runInIO . func)
-- ^^^^^^^ That's the (runInIO . flip runReaderT env) part
-- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ inner
</code></pre>
I copied the snippet from the first chapter, so it’s easier to see how foo2'</code> is passing a callback to withRunInIO</code>, which is what we refer to as inner</code>. That function also receives a callback (here called runInIO</code>), which does exactly the same as the unlift</code> function we pulled out of the UnliftIO</code> newtype in the last chapter.</p>
The basic mechanism is exactly the same: peel off the monad layers, delegating to the UnliftIO</code> instance of each layer, until we eventually reach IO</code>. withRunInIO</code> is just a bit more concise.</p>
Here’s the reader instance with type annotations:</p>
instance MonadUnliftIO m =>
  MonadUnliftIO (ReaderT r m) where
  withRunInIO
  :: ((forall a. ReaderT r m a -> IO a) -> IO b)
  -> ReaderT r m b
  withRunInIO inner =
  ReaderT $ \env ->
    -- withRunInIO
    -- :: ((forall a. m a -> IO a) -> IO b)
    -- -> m b
    withRunInIO $ \runInIO ->
        -- runInIO          |   flip runReaderT env
        -- :: forall a. m a |   :: ReaderT r m a
        -- -> IO a          |   -> m a
      inner (runInIO          .   flip runReaderT env)
</code></pre>
What Now?</h2>
The goal of this post was to understand how unliftio-core</code> works on a basic level. Consider it an intellectual exercise for non-experts (like me) to understand how other people write their Haskell libraries. For information on how this library treats async exceptions or handling the state monad in relation to cleanup handlers, please see the official documentation.</p>
The unliftio</code> library re-exports a lot of modules, none of which I looked at in this post. That’s because once you understand the concept, the implementation of most (all?) of these re-exported modules is fairly mechanical. See for example this random snippet from UnliftIO.Process</code></p>
withCreateProcess ::
   MonadUnliftIO m
  => CreateProcess
  -> (Maybe Handle
    -> Maybe Handle
    -> Maybe Handle
    -> ProcessHandle
    -> m a)
  -> m a
withCreateProcess c action =
  withRunInIO
  (\u ->
     P.withCreateProcess
     c
     (\stdin_h stdout_h stderr_h proc_h ->
      u (action stdin_h stdout_h stderr_h proc_h)))
</code></pre>
Hopefully it’s easy to see the familiar pattern of passing a callback to withRunInIO</code>, where you then have access to an unlift function, thanks to the type class constraint.</p>
^{1</sup>
IO</code> also occurs in positive position.</p>
</div>
^{2</sup>
Technically the liftIO</code> isn’t necessary for this specific use case. Since we’re in the IO</code> monad anyway, liftIO</code> is superfluous here.</p>
</div>
^{3</sup>
foo :: _; foo s = s ++ s</code> gives “Found type wildcard ‘_’ standing for ‘[a] -> [a]’” which we can then use in the signature.</p>
</div>}}}


The Compose Newtype and its Applicative Instance
2019-03-27T00:00:00+00:00
When I went through Haskell From First Principle</em> the first time, I struggled with the Compose</code> applicative instance, which is part of an exercise in chapter 25. This post will give you a quick overview of the Compose</code> data type and then explain how the applicative instance for that type works.</p>
The Compose</code> data type is part of base</a> and allows composing two functors:</p>
> :m Data.Functor.Compose
> let a = Right (Just 2)
> :t Compose a
> Compose a :: Num a => Compose (Either a2) Maybe a
</code></pre>
Any code that starts with ></code> is meant to be run in a repl, such as stack ghci</code> or ghci</code>.</em></p>
Here we took two functors (a Maybe</code> and an Either</code>) and composed them, giving us a new functor! Why is that useful?</p>
> :m Data.Functor.Compose
> let a = Right (Just 2)
> fmap ((+) 2) a
• No instance for (Num (Maybe Int)) arising from a use of ‘+’
</code></pre>
This does not work because fmap</code> is trying to map (+) 2</code> over our Either</code>, which is equivalent to applying (+) 2</code> to Just 2</code> which doesn’t work. However, with the help of Compose</code> we can create a new functor from Either</code> and Maybe</code> which will work with fmap</code>!</p>
> :m Data.Functor.Compose
> let a = Right (Just 2)
> let b = Compose a
> fmap ((+) 2) b
Compose (Right (Just 4))
</code></pre>
The Problem</h2>
In the book, you’re tasked with writing the applicative instance for Compose</code>, which requires at least two functions1</a></sup>: pure</code> and <*></code>.</p>
pure :: a -> fa a
(<*>) :: fa (a -> b) -> fa a -> fa b
</code></pre>
I really struggled with the definition of <*></code>, so I turned to google to find some solutions that I could then reverse engineer. Here’s one possible definition:</p>
instance (Applicative fa, Applicative fb) =>
  Applicative (Compose fa fb) where
  (<*>) ::
	   Compose fa fb (a -> b)
	-> Compose fa fb a
	-> Compose fa fb b
  Compose x <*> Compose y =
	Compose ((<*>) <$> x <*> y)
</code></pre>
Functor types are named fa, fb, and so on. If you see an f, it’s a function, both on the type and the data level. I refer to fa and fb as functors since the applicative type class requires those things to be functors.</em></p>
As is often the case with Haskell, the code is really concise. In a single line we’re dealing with three operators and one of them is the very operator we’re defining for Compose</code> (<*></code>). While that kind of code is a joy to read and write if you’re fluent in all things functor and applicative, it’s a mouthful if you’re trying to improve your understanding of these topics.</p>
So let’s work through the implementation one step at a time!</p>
Something Concrete</h2>
The code below shows a function (+) 2</code> wrapped in a Maybe</code>. We apply it to a value (wrapped in a Maybe</code>) by using <*></code>. Simple enough.</p>
> let a = Just ((+) 2)
> let b = Just 5
> a <*> b
Just 7

	   Just ((+) 2)     Just 5    Just 7
<*> :: fa   (a -> b) -> fa   a -> fa   b
</code></pre>
Let’s up the ante a bit and wrap both the function and the value in another Maybe</code>.</p>
> let a = Just (Just ((+) 2))
> let b = Just (Just 5)
> a <*> b
</code></pre>
This does not work since we can’t just add another layer and expect the original <*></code> to work. After all, its type signature expects function and value to be inside a single functor, not nested in another.</p>
So what’s the #1 solution for manipulating nested stuff in Haskell? fmap</code> all the things!</p>
> :m Control.Applicative
> let a = Just (Just ((+) 2))
> let b = Just (Just 5)
> fmap (<*>) a <*> b
</code></pre>
We map <*></code> over a</code>, meaning we partially apply <*></code> to the Just ((+) 2)</code> inside a</code>. We then take that partially applied function (which is still inside the functor a</code>) and apply it to the Just 5</code> in b</code>. Please go ahead and open a repl now and play around with that code, it can do wonders for understanding stuff like that.</p>
Something Abstract</h2>
But how does the fmap</code> knowledge from the last paragraph help us make sense of the instance code?</p>
instance (Applicative fa, Applicative fb) =>
  Applicative (Compose fa fb) where
  (<*>) ::
	   Compose fa fb (a -> b)
	-> Compose fa fb a
	-> Compose fa fb b
  Compose x <*> Compose y =
	Compose ((<*>) <$> x <*> y)
</code></pre>
The first part (<*>) <$> x</code> written without infix notation and fmap</code> instead of the operator is fmap (<*>) x</code>. We map the function <*></code> over x</code> and x</code> has the type fa fb (a -> b)</code>. We therefore apply <*></code> to the fb (a -> b)</code> inside x</code>. Check out the commented code below, which hopefully makes things clearer.</p>
instance (Applicative fa, Applicative fb) =>
  Applicative (Compose fa fb) where
	(<*>) ::
		  Compose fa fb (a -> b)
				 --  ^^^^^^^^^^^
				 -- This is the first argument to <*>
	   -> Compose fa fb a
	   -> Compose fa fb b
	Compose x <*> Compose y =
		-- fa' :: fa (fb a -> fb b)
		--            ^^^^ The 2nd argument to <*>
		let fa' = fmap (<*>) x
			   -- ^^^^ mapping <*> over x means applying
			   -- <*> to the content of x
		in ???
</code></pre>
The <*></code> only needs its 2nd argument now, which is a functor with a value inside it. And we have something like that inside</strong> our y</code> (y</code> is fa fb b</code> and therefore the missing argument to <*></code> is the fb b</code> part inside the fa</code>). How can we apply a function inside a functor to a value inside a functor? <*></code>! And that’s how we arrive at the 2nd part:</p>
instance (Applicative fa, Applicative fb) =>
  Applicative (Compose fa fb) where
	(<*>) ::
		  Compose fa fb (a -> b)
	   -> Compose fa fb a
	   -> Compose fa fb b
	Compose x <*> Compose y =
		let fa' = fmap (<*>) x
		in Compose $ fa' <*> y
</code></pre>
This is not so different from how we used <*></code> in “Something Concrete”, just that both arguments have an additional level of nesting. Aligning the type signatures of <*></code> (top) and Compose x <*> Compose y</code> (bottom) helps with visualising the similarities. It’s the exact same operation one level deeper for both arguments.</p>
   fb (a -> b) ->    fb a ->    fb b
fa fb (a -> b) -> fa fb a -> fa fb b
</code></pre>
Quick recap:</p>

fmap (<*>) x</code>: Partially apply <*></code> the contents of x</code>, which gives us a functor holding a partially applied function.</li>
fa' <*> y</code>: Fully apply the <*></code> inside fa'</code> (!) to the contents of y</code> through another use of <*></code>.</li>
</ul>
The hackage implementation</h2>
The implementation of the applicative instance using <*></code> and a mix of infix operators requires some mental gymnastics. On hackage</a> however the instance uses liftA2</code>, which does a much better job of communicating the essence of what’s going on. Here’s an example of how liftA2</code> works, and where it’s compared to our use of liftA</code> from above.</p>
> let a = Just (Just ((+) 2))
> let b = Just (Just 5)
> fmap (<*>) a <*> b
Just (Just 7)
> liftA (<*>) a <*> b
Just (Just 7)
> liftA2 (<*>) a b
Just (Just 7)
</code></pre>
As you can see, liftA2</code> leads to the same result but is a bit more concise and expressive in this case. We can use liftA2</code> to conveniently apply <*></code> to the contents of the two functors in the two Compose</code> types.</p>
Edit</h2>
Thanks to u/Syrak</code> from reddit for reminding me that liftA</code> and fmap</code> are pretty much the same. I edited the post so that liftA</code> is only used at the very end. See also his comment</a> for more insights!</p>
^{1</sup>
Technically the minimal definition of applicative requires pure</code> and either <*></code> or liftA2</code></p>
</div>}
Getting a Raspberry Pi 5

Advent of Code 2024 Day 6: Janet Streams Using Fibers/Coroutines

Advent of Code 2024 Day 2: Parsing Expression Grammars

Dark/Light Switching for the Terminal

Heist: Haskell Templating Woes

Two Years of Nix & Home Manager

Summary</h2> After utilizing Nix and HM to manage both a MacOS and a NixOS machine for a solid two years, I can confidently say that the experience has been incredibly pleasant overall. Reflecting back, the major advantages that stood out, and continue to do so, include:</p>

Streamlined Package Management</h3> When I switched from Nix to MacPorts, it became evident how many package managers I relied on:</p> MacPorts</li> Fish plugin manager</li> Vim/Neovim plugin manager</li> Visual Studio Code plugin manager</li> tmux plugin manager</li>

What didn’t work so well</h2>

Building a Website With Haskell & Nix

Deep Dive</h2>

Nix</h3> According to tokei</code> this project has 1206 lines of Nix code, which fall into three categories: developer environment, building parts of the application, building and deploying the NixOS image that runs on a digital ocean droplet.</p>

Ramda vs. Vanilla Javascript

Exploring the Dhall configuration language

Understanding unliftio

The Compose Newtype and its Applicative Instance
