In this post we're going to set-up Leiningen so that we have some standard development tools every time we start the REPL. We'll also cover the basics of Leiningen and the REPL - just enough to be dangerous!
In the previous post we set-up Clojure with Lein and Rebel Readline to create a solid REPL user experience. If you haven't read that check it out now ... I'll wait.
Back? ... lets get started then.
The most important concept to understand with Leiningen is a 'project'. Every project in Lein consists of a set of directories that contain all the assets for the application. When Leiningen is invoked it looks for a file called project.clj where all the settings for the project are stored: it looks for that file in the current working directory. When you create a Clojure project with Leiningen it will create a standard project structure for you.
This is just enough to get going - for all the details go through the tutorial. You can see the tasks that Lein knows about by calling it:
$ lein Leiningen is a tool for working with Clojure projects. Several tasks are available: ancient Check your projects and profiles for outdated dependencies/plugins. bat-test Run clojure.test tests. change Rewrite project.clj with f applied to the value at key-or-path. ... <lots more> ...
Each task has help:
$ lein help upgrade
The most important ones are:
Use the new task to create a new project with a standard project layout: it will create the project directory and a set of standard locations for source and tests. It also creates a project.clj for the project that we can then customise for the specific project.
lein new app <your-project-name>
Run tests, the default is that these are in test/<namespace>/core_test.clj. Clojure comes with a test framework.
lein test [tests]
Using the alias we set-up last time to launch the rebel-readline REPL.
lein rebl lein repl ;; the standard repl
Leiningen is extensible through plugins. There are plugins for all sorts of developer experience tools such as testing, formatting, git integration and build automation.
Since everything in Lein is normal Clojure code plugins are distributed through Clojars . A plugin's page gives you some sense of it's popularity, as well as instructions on how to install it.
As an example we're going to install Lein-Ancient. It's the most important plugin, practically universal, as it checks and updates dependencies. Clojure projects often have large sets of dependencies, so automation is important.
To install a plugin we add it to one of the configuration files and Leiningen will download and configure it. Plugins can be loaded either globally (~/lein/profiles.clj), or for a particular project (<project root>/project.clj).
Lein Ancient is a plugin we want available all the time so we'll put it into ~/.lein/profiles.clj. The specific Clojars page tells us what to add:
[lein-ancient "0.6.15"]
It's added in the :plugins section, like this:
{:user
{:plugins [; ... other plugins ...
[lein-ancient "0.6.15"]
]
If we run lein deps or start the REPL it will download and install the plugin.
Lein Ancient can check projects (by looking in the project.clj), and the global profiles (looking at ~/.lein/profiles.clj) for out of date dependencies. To see the full help do lein help ancient, but the main commands are:
lein ancient check ;;default check for outdated project dependencies lein ancient check-profiles ;;check profiles for outdated dependencies lein ancient upgrade ;;upgrade project dependencies lein ancient upgrade-profiles ;;upgrade profile dependencies
As we've seen Plugins change Lein's behaviour, adding commands and altering the REPL start-up: we'll look at this 'middleware' concept later. It follows that you'll want different plugins depending on the task, that's the purpose of Lein Profiles.
Profiles load librarys when we're performing a Lein task (such as starting the REPL). Last time we put some settings into the :user section of our global ~/.lein/profiles.clj so that we could start rebel-readline. This was an example of using a Lein profile - specifically the user profile. We're going to continue developing our user profile with general developer experience capabilities that are loaded every time we start the REPL.
Profiles let you change the settings that Lein uses when a command like lein rebl is run. They are maps of options that are applied to tasks. They can be configured in the following locations:
1. ~/.lein/profiles.clj 2. <project root>/project.clj 3. <project root>/profiles.clj
The ordering is that the project settings over-ride the user-wide settings specified in ~/.lein/profiles.clj. The project specific profiles.clj will over-ride the project project.clj.
We can list all profiles with lein show-profiles:
$ lein show-profiles base debug default leiningen/default leiningen/test offline repl uberjar update user
The list above shows the default profiles. We can create our own profile and use them with a task. The more common step is to configure the profiles that are used for certain tasks.
The user profile is the place to store any settings or tools that we want every time we run a Lein command such as launching a REPL. This is a global capability so it's stored in ~/.lein/profiles.clj.
When starting out I found the naming around this area confusing. You'll sometimes find instructions to put plugins or settings into the user profile and sometimes into the dev profile. This is two separate things:
To avoid collision you should not define a :user profile in a project - if you do specify them in two locations then you'll get a conflict because profiles are not merged. Side-note: there is a way to merge profiles but it's quite advanced and I'm ignoring it for this post.
These are the plugins that I have installed in my ~/.lein/profiles.clj for REPL-based development:
{:user
{:plugins [; ... other plugins ...
[lein-ancient "0.6.15"] ;check dependencies
[lein-pprint "1.3.2"] ;pretty print lein dependencies
[metosin/bat-test "0.4.4"] ;test runner
[lein-try "0.4.3"] ;easy to try new libraries
[io.aviso/pretty "0.1.37"] ;make stack traces pretty
]
:middleware [io.aviso.lein-pretty/inject ;starts pretty during REPL start-up
]
:dependencies [
[com.bhauman/rebel-readline "0.1.4"] ;nice REPL
[io.aviso/pretty "0.1.37"] ;formatting of exceptions
[mvxcvi/puget "1.2.1"] ;colour print data
[expound "0.8.4"] ;improve error messages
[expectations/clojure-test "1.2.1"] ;library for testing
[org.clojure/tools.namespace "1.0.0"] ;reload
]
:injections [
;colours for stack traces in the REPL
(require 'io.aviso.repl)
]
:aliases {"rebl" ["trampoline" "run" "-m" "rebel-readline.main"]}
}
}
As you can see whether a dependency is a Lein plugin or a library it's handled in the same way. We look it up on Clojars, get the version and add it to the right section.
We discussed Lein Ancient earlier, and Rebel-readline in the previous post.
pretty is a plugin that changes the REPL's behaviour to make formatting nice using ANSI colours and vastly improves error messages. puget provides a ANSI colour pretty printer, it's used by pretty.
Note that pretty has to be added to both the plugin and the dependencies sections. It's also an example of a middleware as it injects itself in the REPL start-up process.
Lein pprint prints a representation of the project map. It's really useful when you want to understand how Leins global and local settings are coming together. After it's installed there's a new task: lein pprint.
Lein-try lets you try out any Clojure library using the command lein try <library>.
Bat-test is an example of a plugin that itself brings multiple plugins together! It brings some nice testing tools (ef-test, cloverage) and for me enables a test auto-runner: lein bat-test auto
expectations/clojure-test this makes the expectations library compatible with the standard clojure test. It simplifies testing a lot.
We now have some global tools and settings (in the user profile), but what about project specific settings?
That's the job of the dev profile. The dev profile in the project project.clj keeps any settings or plugins just for that project. When Lein runs a task the dev profile takes precedence over the user profile. That's because project local settings take precedence.
We can load plugins or librarys in the same way as shown earlier. The new element is we can load libraries so that they're immediately available into the REPL environment (Rebel Readline or Vim for me).
This is done by telling Lein about a Clojure file that will be loaded automatically as the REPL is started. Technically, the file can be anywhere you'd like in the project hierarchy. Luminus uses the concept of an env directory within each project. Environment specific files are stored here, for example env/dev/, env/test/ and env/prod/.
Lets say that we've created a project called test-rebel-readline, in the projects project.clj we add the following:
:profiles {:dev
{
:dependencies [; ... dev dependancies ...
]
:source-paths ["env/dev"] ;additional source path
:main user/-main ;which namespace and function to start in
}
}
In the env/dev/ directory we add a user.clj which has the following:
(ns user
(:require
[clojure.repl :refer :all]
[clojure.pprint :refer (pprint)]
[puget.printer :refer [cprint]]
[clojure.tools.namespace.repl :refer [refresh refresh-all]]
[clojure.test :refer :all :exclude [run-tests]]
)
)
When the project dev profile runs it looks in the env/dev directory and loads the user.clj. If you don't specify a namespace with :main then you'll be put into the user namespace.
As you can see we're loading a bunch of tools into the REPL. The easiest one to test is to see if the clojure.repl namespace is loading - if it is then (doc print) will print the documentation for the print function.
With the tooling all loaded up we're ready to look at the REPL experience. We've automatically loaded the clojure.repl namespace so the main REPL functions are available:
;; shows information about the function user=> (doc print) ;; shows the source code for the function user=> (source println) ;; find any and all docs with print in them - don't use it much user=> (find-doc "print") ;; find any Var with 'per' in the name user=> (apropos 'pea)
The apropos function is for finding any var that matches the search in any loaded namespace. Here's an example:
user=> (def pear 4) ;;=> #'user/pears user=> (apropos 'pea) (clojure.core/repeat clojure.core/repeatedly rewrite-clj.reader/read-repeatedly user/pears)
As you can see it does find our pears Var, as well as some other functions that match.
To see all the functions and vars available within a namespace use the clojure.repl.dir function:
user=> (dir user) user=> (dir clojure.string)
Errors and Stacktraces in Clojure just aren't very nice - there's no getting around it. The default is that they spew lots of information which is very opaque.
It's definitely a big wart on the language and makes it difficult to understand what's happening as you're learning. It's a well-known critique of Clojure, and it's right on the money - but there's not much chance of it changing so lets move on!
Pretty is the biggest improvement I know of, it does a great job improving stacktraces. It shortens the stacktrace and tries to show you the root cause clearly. Here's an example:
;; create an error - pretty trys to shorten it for you user=> (/ 1 0) java.lang.AirthmeticException: Divide by zero ;; *e stores the last exception user=> *e #error { :cause "Divide by zero" <... full stacktrace ...> ;; prints a stacktrace of the exception with the pst function user=> (pst) clojure.core/eval core.clj: 3214 ... user/eval7349 REPL Input ... java.lang.ArithmeticException: Divide by zero
The most useful command to know is pst which prints the last stacktrace. The formatting is changed by Pretty to make it easier to understand.
The full stacktrace in all it's glory is stored in *e, so it's accessible if we want to know more detail.
Clojure takes a while to start and the more libraries and tools your project adds the longer it takes. The most common workflow is to avoid having to restart the REPL by reloading functions and namespaces. Imagine that you've written a new version of a function and you want to load it into the REPL.
A helper library tools.namespace has an improved refresh function:
; it was defined in the user.clj higher, but can also be loaded in the REPL like this user=> (require '[clojure.tools.namespace.repl :refer [refresh]]) user=> (refresh) :reloading (test-rebel-readline.core test-rebel-readline.core-test user) :ok
According to the documentation the refresh function scans source code directories. When you change some files and then refresh it will only reload the namespaces you've changed.
The more sophisticated version is a workflow called 'Reloaded' by Stuart Sierra that is popular when you have complex applications - see My Clojure Workflow, Reloaded. There's also a set of tooling around this concept.
There's lots of good content on Leiningen and the REPL.
At this point we have the best core Leiningen and REPL experience I know of. It's a 'pure REPL' experience where we work directly in the REPL. It's perfect for learning Clojure and playing with small functions.
For a full development experience we still need to connect an editor: we'll cover that in the future.
In this post we've learnt Lein's basics and how to configure it through plugins. We've also gone through how to configure a project dev profile and how to use a user.clj for a project specific developer experience. Finally, we looked at the REPL experience.
There's lots of other plugins and libraries to try, hopefully the ones I've listed are a good start! Which important ones did I miss?