Clojask: Inviting Data Scientists to Distributed Computing

Abstract

Clojask is a distributed dataframe with a focus on usability and scalability. On one hand, Clojask is simple to use so that data scientists without any distributed systems experience can use Clojask immediately. The API design is inspired by R's data.table and SQL, so the learning curve is flat. On the other hand, Clojask is optimized for larger-than-memory datasets. Memory overflow will not be a problem even for tasks with massive datasets. Both technical considerations are determined to attract and benefit users with prior data science experience to Clojure. In our session, we would like to cover topics such as a functionality walkthrough (with reference to R data.table and SQL), comparisons with Dask (in Python) and Spark as well as what Clojask can bring to the Clojure data science community.