How Clojure REPL works

If you are using Clojure REPL, there will be times you want to adjust the REPL behavior, the only way will be to modify the source code of Clojure. Clojure is open source, you can build it from source and run it in 1 minutes.

This post illustrate some key points in Clojure's REPL. The source code is actually very simple and straightforward, but a manual may be easy to get start from.

Build Clojure from sources

This post uses Clojure 1.6, download the package and unpack it, the project is built with Maven. You may want to skip the test, the command should be

 
mvn -Dmaven.test.skip=true package
 

The Clojure jar will be generated in target folder, to run it with the command

 
@echo off
 
set CLOJURE_DIR=C:\tmp\clojure1.6\target\clojure-1.6.0
set CLOJURE_VERSION=1.6.0
 
set CLOJURE_JAR="%CLOJURE_DIR%/modules/*";
 
if (%1) == () (
     :: Start REPL -verbose:class
     java -server -cp .;./*;%CLOJURE_JAR% clojure.main  -r
) else (
     :: Start some_script.clj
     java -server -cp .;%CLOJURE_JAR% clojure.main %1 -- %*
)
 

The entry point located in clojure1.6\src\clj\clojure\main.clj, and some related files: clojure1.6\src\clj\clojure\core.clj, clojure1.6\src\jvm\clojure\lang\LispReader.java.

Where the REPL waiting input

At the end of every REPL loop, the prompt is emit and flush to the terminal, then goes to the beginning of the loop with recur

Then the REPL will be blocked in the skip-whitespace function. The (.read s) will be blocked until you press Enter in the REPL.

 
defn skip-whitespace
  [s]
  (loop [c (.read s)]
    (cond
     (= c (int \newline)) :line-start
     (= c -1) :stream-end
     (= c (int \;)) (do (.readLine s) :line-start)
     (or (Character/isWhitespace (char c)) (= c (int \,))) (recur (.read s))
     :else (do (.unread s c) :body)))
 

This function is what will happen after you press Enter, it will skip comments and whitespace and make sure there are expression body needs to be read and parsed. The rest will be handled by LispReader.

The *in* stream

In the REPL, everything are read from an input stream, there is a global variable *in* to hold the input stream object defined as below

 
final static public Var IN =
        Var.intern(CLOJURE_NS, Symbol.intern("*in*"),
                   new LineNumberingPushbackReader(new InputStreamReader(System.in))).setDynamic();
 

The lisp reader is a simple parser which read from the PushbackReader represented by *in*. When you don't evaluate any expressions, the REPL is blocked by the read method of the PushbackReader until you press Enter.

This provides a possibility: before the stream is sent to LispReader, we can modify the stream with the unread method of PushbackReader. It can do something that Clojure itself can not do.