Core.async for simple user-interaction

Alex Miller gives an excellent example on how to make rock-paper-scissors with core.async which made me want to try it on my mekanikmaskin - a computer-aided learning and examinating tool.

The cool thing about core.async is that it makes sort-of a reactive programming possible. Put something in a channel, and watch the program take care of it. Almost like REPLs waiting inside functions wherever you like.


And yes, it's a terrible mixture between printlns and the async constructs, but the way to create a more advanced interaction is for another day.

Project.clj superpowers with Alembic

Hugo Duncan (Pallet etc) have made it possible to dynamically add new Maven/Leiningen dependencies from the REPL through Alembic.

This opens up from starting a repl with just Clojure and keep adding dependencies all night long.

As a courtesy to you, dear reader, a current project.clj of mine have the following deps so you can try them out for yourself, in the live-running repl.

Some of my common Project.clj dependencies

[org.clojure/clojure "1.5.1"]
[org.clojure/tools.logging "0.2.6"]
[org.clojure/data.csv "0.1.2"]
[org.clojure/data.json "0.2.2"]

The Clojure language (which should already be available in the repl), functions for logging, parsing comma-separated values from exports and parse JSON from REST-apis.

[org.clojure/google-closure-library-third-party "0.0-2029"]
Needed for ClojureScript code compilation. Probably outdated and I should switch to Pedestal instead.

[clj-time "0.5.0"]
A wrapper around Joda-time, the sane way to handle time, date and intervals on the JVM.

[clj-http "0.7.2"]
An http-client based on Apache Commons

[ch.qos.logback/logback-classic "1.0.11"]
The best working logging library IMHO.

[incanter/incanter-core "1.5.1"]
[incanter/incanter-io "1.5.1"]
[incanter/incanter-charts "1.5.1"]
Incanter is a R-like package for Clojure. There are functions for various statistics, matrix calculations, not-super-cool-but-convenient visualization through JFreeChart, readers for various data sets and more.

[enlive "1.1.1"]
HTML Templating library based on transforms.

[domina "1.0.0"]
ClojureScript-library for manipulating the DOM.

[com.datomic/datomic-free "0.8.4020"
 :exclusions [org.slf4j/slf4j-nop          
              org.slf4j/slf4j-log4j12]]
A convenient, scalable, relational database that is built in Clojure keeping many of Clojure's idioms while getting the Durability in Clojures ACI-persistent datastructures. The exclusions is to make sure the logging from Datomic not collides with logback.

[compojure "1.1.5"]
A convenient library to make ring-handlers for specified routes. If you come from PHP-world this is like where you put the "filenames" and/or the rewrite rules, only that it's made right from the beginning. 

[ring "1.1.8"]
The servlet-abstraction library for clojure.

[com.keminglabs/vomnibus "0.3.2"]
Small useful things, like the ColorBrewer colors.

[com.keminglabs/c2 "0.2.2"]
I try to make visualizations with this one instead of D3, but it's less mature and I'm not skilled enough in the toolchain to get as much from it as from D3 right now. I have not given up yet!

[ring-mock "0.1.5"]
Mocking is a way to create a fake instance of something, in this case it allows a ring handler function to be called to verify that the function returns excepted results, all this without starting a web-server, eliminating the possibility to have collisions in port-allocation etc.

[clj-pid "0.1"]
A .pid contains the Process ID (an integer) as a string. This can be used to verify the process is running, and for kill it by a shellscript.

[schejulure "0.1.4"]
A cron-like scheduling library based on the Java concurrent scheduling libraries and some core.logic.

Clj-Liblinear (thanks Kevin)

I had some problems with "typed" ragged arrays when I tried to wrap Lib-linear in Clojure. Kevin Lynagh and Benedikt Waldvogel had already solved them in clj-liblinear, thanks!

Now onto some batching for determining nice parameters for our tests.

Quick distinct set generator

In one application I needed to create quite many sets of distinct numbers in a limited range.

This is done quickly by doing a knuth-shuffle of an array with all the numbers in the range and do the random swapping as many steps as the size of the requested set.

Here's a typed and thread safe version - no reflections gives a super quick execution. The locking macro makes sure we don't mix everything up by parallel threads.

(let [vektor-length 400
      set-size 20
      shuffle-seed (long-array (shuffle (range vektor-length)))]

  (defn shuffle-n!
    "swaps the n first with any other in vector,
  (see knuth shuffle)
array locking has to be done by the caller, otherwise
we could have leaks -> duplicates"
    [^longs arr n]
    (dotimes [idx n]
      (let [shuffle-with-idx (rand-int vektor-length)
            a (aget arr idx)
            b (aget arr shuffle-with-idx)]
        (aset arr idx b)
        (aset arr shuffle-with-idx a))))

  (defn gen-distinct-set []
    (locking ^longs shuffle-seed
             (shuffle-n! shuffle-seed set-size)
             (let [new-distinct-set (long-array set-size)]
               (dotimes [idx set-size]
                 (aset new-distinct-set idx (aget ^longs shuffle-seed idx)))
               new-distinct-set))))


A test of this algorithm rendered a million sets in less than a second, which was more than sufficient for my application, otherwise my next step would have been to dig into thread-local constructs in flatland/useful thread-local macro, where every thread could have it's own seed vector.

(I don't check for duplicates, since they are unlikely enough for my application).

Don'ts: Git hooks as CI/deploy tool

GIҬ

I want to get Heroku-like functionality for deploying new version - just git push and you're done. Crazy sexy cool.

I did try with git hooks in various ways, but since I run the server as a nohup jetty-process (I know) and have to kill that in brutal ways, cannot be sure on the merging of the branches since I had the git repo checked out (use a intermediate bare repo, please) I could not get it work. It was also really hard to know what was happening.

The search light is now against lein-war (for running in tomcat7) and some nice pallet solution.\

Git hooks is awesome for other things, but they are no system deployment/ continuous integration tool.

Update: I'm now a happy Jenkins user. Easy to get started with!

merge-with to the rescue

I'm working on coloring a map with different colors, that sometimes overlap.

After deep though I found out that I could have colorings in maps and then merge the maps somehow. This doesn't work perfectly, but it's a good start. After fiddling around with home-brewed solutions I looked in core.clj for merge in the source code and found merge-with just below. Perfect match.

A short example of the power of merge-with:


(def scandinavia (zipmap ["sweden" "norway" "denmark" "aaland" "iceland" "finland" "greenland" "fareoes"] (repeat :yellow)))
=>{"fareoes" :yellow, ...

(def swedish-speaking (zipmap ["sweden" "aeland" "finland" "ukraine" ] (repeat :red)))
=>{"ukraine" :red, ... * see below for Swedish language in Ukraine

(defn blend [a b] :orange)

(merge-with blend scandinavia swedish-speaking)

{"ukraine" :red, 
"fareoes" :yellow, 
"greenland" :yellow, 
"finland" :orange, 
"iceland" :yellow, 
"aaland" :orange, 
"denmark" :yellow, 
"norway" :yellow, 
"sweden" :orange}

As you can I cheated quite extensively with the blend function, but in this case it doesn't matter, since we're always blending yellow and red if we are blendning.

Merge-with uses the function given (blend) if two keys are colliding when merging. If you want to sum to maps, use (merge-with + {:x 1 :y 2} {:x 3}) and get {:x 4 :y 2}.

* And yes, there are some really old ladies speaking Swedish in Ukraine in the Gammalsvenskby.

Minimal C2 bind! example

 C2 is a very sane vizualisation library for both Clojure and ClojureScript.

I had a hard time grasping bind! and unify, so I wanted to make the smallest example I could think of: generate a blue svg circle.

From O'Reilly's excellent introduction to SVG (available online) I found that the XML for an SVG circle would be something like:

<svg width="400px" height="400px">
 <circle cx="100" cy="100" r="45" style="fill: 'blue';"/>
</svg> 

I did once spend unreasonably long time to try to reach the DOM of an embedded SVG image, so from now on I embed the SVG DOM root in my html document, like this document with an empty SVG tag with id "bubbles":

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
 <head>
  <title>SVG bubble</title>
 </head>
 <body>
 <div>
  <svg id="bubbles"></svg></div>
 </div>
 <script src="/cljscode.js"></script>
 </body>
</html>

C2 transforms tags like Hiccup do, but you can say where you want to add it with the macro c2.util/bind! (works a bit like in enlive).

And some very minimal ClojureScript code:

(ns c2test.core
 (:require  [c2.core :as c2core])
 (:require-macros [c2.util :as c2util]))

(defn ^:export c2bubble [x y r]
  (c2util/bind! "#bubbles"
                [:svg#bubbles {:style  {:display "block" :margin "auto" :height 200 :width 200}}
                 [:circle {:cx x :cy y :r r :style { :fill "blue"}}]]))

By opening a browser console (for javascript) I can set coordinates of a blue bubble with a command like

c2test.core.c2bubble(20,50,30);

Now on to world domination.