>Programmers can expose C++ classes as well as functions and class methods with a single line of code that provides the Clasp name and a pointer to the function/method.
It would be really shiny to have a cross-platform GUI library in CL without grief. I've prodded that space repeatedly in the last 6 years and simply have not come away happy about any of it except LispWork's library.
A faster Clasp compiler is coming soon – the current Clasp compiler generates slow native code that is about 100x slower than highly tuned Common Lisp compilers like Steel Bank Common Lisp.
No mention of the expected speedup, but 100x is a huge performance problem.
The reason is that Clasp doesn't do Lisp language level optimizations like escape analysis yet. All bindings are stored on the heap and the stack/registers are underutilized. LLVM is a great library for implementing C and C++ but more work needs to be done to support Lisp features like closures and first-class functions. We are working on that now. The goal was first to "make it correct" and now we will "make it fast". Once we have a faster compiler (give us a couple of months) I don't see why it couldn't approach the speed of SBCL (a tall order).
Glad to hear it. I've been wondering when new languages would be developed on top of LLVM and it's starting to happen (Julia, now this). C++ is getting a lot of more advanced features, but the syntax is so clunky compared to something like Python for example. I also like the notion of easy C++ library use for things like QT.
> Considering that SBCL is among the fastest programming language implementations there is...
Source for this? I'd love to find a really fast lisp or scheme. Hadn't heard SBCL was particularly fast, and the alioth benchmarks don't show anything special there.
I'm not too worried, since a) that makes it about as fast as cpython and b) there is a lot of low-hanging fruit. There is work being done on integrating the Cleavir compiler which does a number of tricks. In particular escape analysis and lambda lifting should both help a lot on certain types of code.
Essentially, LLVM does tons of low-level optimizations but Clasp does few high-level, CL-specific ones. SBCL, on the other hand, mostly does CL-level optimizations and few low-level ones.
Yes, LLVM is awesome. Clasp uses the inlining LLVM provides to inline C++ code within Common Lisp code by generating LLVM-IR bitcode files from C++ source and linking them with Clasp generated bitcode files and then running them through LLVM module and function optimization passes. I'm really excited about combining the CL specific optimizations with the ones that LLVM provides - stay tuned.
Almost certainly can't. The closest you can get is using clicc or ECL or something similar to generate C code, and then compile that with emscripten.
This invokes llvm at runtime, and AFAIK emscripten isn't ported to emscripten.
FWIW, I've tried other lisps under emscripten:
Most lisps generate machine code, store them in RAM, and then execute that RAM. This is not possible under emscripten.
Clisp is a good candidate, since it's byte-code interpreted rather than generating machine code, but clisp makes so many assumptions about how the machine works (in particular it strongly wants a C style stack and does manual stack-pointer manipulation). I actually got fairly far into the bootstrap process under emscripten, but the minimal lisp interpreter it compiles generated bizarre errors.
I disagree. Clasp could do this because it compiles Common Lisp to LLVM-IR bitcode files (using COMPILE-FILE) as well as directly to native code using LLVM's MCJIT engine (using COMPILE). emscripten (https://github.com/kripken/emscripten) says that it compiles LLVM-IR to JavaScript. I haven't used emscripten, but I believe everything I read on the internet :-) and thus Common Lisp --[Clasp]--> LLVM-IR bitcode files --[emscripten]--> run within browsers.
Sorry to go offtopic but since the you mentioned emscripten I though I'd ask.
Can you explain what emscripten is used for? I'm fully aware of what it does, just not why someone would want to use it. How do you use things like jQuery or access window or document from C and then compile it to js? Or is emscripten specifically for js that doesn't interact with the DOM?
It works really well. It's a subset of CL that it can compile though so you'll miss out on CLOS, restarts, etc. However the subset does include macros so it's serviceable.
I don't see why we couldn't do this. emscripten (https://github.com/kripken/emscripten) says it takes LLVM-IR to JavaScript. The Clasp (Common Lisp) function COMPILE-FILE compiles to LLVM-IR bitcode files. If you have some time and you would like to do this - hit me up with an email - I'd love to make this happen.
You'll probably run into all the same problems other Common Lisps run in to when they try to shake out the compiler from their runtime, namely that often you really do want to create and run new code at runtime (i.e. EVAL but much more often (compile nil '(lambda () ...)). A common example in CL is creating CLOS dispatch functions with specialized parameters to speed up generic dispatch, which I believe most CLOS implementations do at runtime (certainly SBCL/CMUCL's PCL does). You'll wind up needing to include your compiler, Emscripten itself, and all of their dependencies (LLVM libraries?) which will be quite a huge pile of JavaScript. The alternative is to include a CL interpreter but that has obvious speed problems.
Is this project something that you're doing to support your own research goals in computational chemistry? If so, can you share how this project fits into the bigger picture?
SBCL doesn't generate LLVM code. My primary goal was Common Lisp with C++ interoperation. It seemed easier at the time to start from the ECL Common Lisp code base and write a new C++ core and a Common Lisp interpreter that always interoperated with C++. As I wrote the compiler and expanded the system I maintained C++ interoperation all the time. There were a hundred problems that I had to discover and solve along the way to maintain a Common Lisp that interoperated with C++. You can get some level of interoperation between ECL and C++ up in a weekend but it won't work with C++ exceptions and RAII and there are dozens of other problems. In retrospect I don't think I would have gotten here starting from ECL because I never really understood the ECL code.
Yes, the Memory Pool System by Ravenbrook (https://www.ravenbrook.com/project/mps/) is a proper, moving garbage collector. It uses precise GC on the heap and conservative GC on the stack, as does the garbage collector in Steel Bank Common Lisp on x86 chips. I need it because I need my code to run on 100,000 CPU supercomputers with a controlled memory footprint to develop organic nano machines (seriously).
I would point out that MOCL https://wukix.com/mocl pretty much provides this functionality only with full Common Lisp compatibility. It compiles the CL to C and adds methods to expose your functions to Obj-C / C code (if you want). Works just fine. Builds with Xcode, which as I understand, does everything on/via LLVM.
mocl is different from many other implementations: it is a batch compiler for whole programs to C. The generated code does contain a limited evaluator and no compiler. Some dynamic features are gone...
> Clasp exposes the Clang AST library and the Clang ASTMatcher library. This allows programmers to write tools in Common Lisp that automatically analyze and refactor C++ programs. Clasp can be used to automatically clean-up and refactor large C++ codebases!
Yes, I'll post more on this and provide example code in the coming weeks on the blog. It's really exciting and it's something that Clasp can do right now even though the Clasp compiler isn't a highly optimizing one yet. Google uses automated Clang refactoring to clean up the googleplex (100 megaLOC). Clasp puts those capabilities into everyones hands now.
Package: clasp
Version: 3.1.0-1
Description-en: conflict-driven nogood learning answer set solver
clasp is an answer set solver for (extended) normal logic
programs.
It was also already taken by a device for fastening things. And my name was already taken by at least three people before me. Somehow we all get along.
>Programmers can expose C++ classes as well as functions and class methods with a single line of code that provides the Clasp name and a pointer to the function/method.
Hellooooo Qt without Smoke bindings.
It would be really shiny to have a cross-platform GUI library in CL without grief. I've prodded that space repeatedly in the last 6 years and simply have not come away happy about any of it except LispWork's library.
Sounded good until I saw this:
A faster Clasp compiler is coming soon – the current Clasp compiler generates slow native code that is about 100x slower than highly tuned Common Lisp compilers like Steel Bank Common Lisp.
No mention of the expected speedup, but 100x is a huge performance problem.
The reason is that Clasp doesn't do Lisp language level optimizations like escape analysis yet. All bindings are stored on the heap and the stack/registers are underutilized. LLVM is a great library for implementing C and C++ but more work needs to be done to support Lisp features like closures and first-class functions. We are working on that now. The goal was first to "make it correct" and now we will "make it fast". Once we have a faster compiler (give us a couple of months) I don't see why it couldn't approach the speed of SBCL (a tall order).
Glad to hear it. I've been wondering when new languages would be developed on top of LLVM and it's starting to happen (Julia, now this). C++ is getting a lot of more advanced features, but the syntax is so clunky compared to something like Python for example. I also like the notion of easy C++ library use for things like QT.
Considering that SBCL is among the fastest programming language implementations there is, 100x is actually not terrible for a first version.
> Considering that SBCL is among the fastest programming language implementations there is...
Source for this? I'd love to find a really fast lisp or scheme. Hadn't heard SBCL was particularly fast, and the alioth benchmarks don't show anything special there.
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
Edit: actually SBCL stacks up alright against languages like Go or Rust in the alioth benchmarks, so maybe that's what you had in mind.
11 replies →
I'm not too worried, since a) that makes it about as fast as cpython and b) there is a lot of low-hanging fruit. There is work being done on integrating the Cleavir compiler which does a number of tricks. In particular escape analysis and lambda lifting should both help a lot on certain types of code.
Any weekend compiler project can be faster than CPython.
6 replies →
http://log.irc.tymoon.eu/freenode/lisp?around=2014-09-25T18:...
Essentially, LLVM does tons of low-level optimizations but Clasp does few high-level, CL-specific ones. SBCL, on the other hand, mostly does CL-level optimizations and few low-level ones.
Yes, LLVM is awesome. Clasp uses the inlining LLVM provides to inline C++ code within Common Lisp code by generating LLVM-IR bitcode files from C++ source and linking them with Clasp generated bitcode files and then running them through LLVM module and function optimization passes. I'm really excited about combining the CL specific optimizations with the ones that LLVM provides - stay tuned.
In his defense, SBCL is really fast...
This is extremely exciting, particularly if it can be made to work with emscripten. Common Lisp in the browser, here we come!
Almost certainly can't. The closest you can get is using clicc or ECL or something similar to generate C code, and then compile that with emscripten.
This invokes llvm at runtime, and AFAIK emscripten isn't ported to emscripten.
FWIW, I've tried other lisps under emscripten:
Most lisps generate machine code, store them in RAM, and then execute that RAM. This is not possible under emscripten.
Clisp is a good candidate, since it's byte-code interpreted rather than generating machine code, but clisp makes so many assumptions about how the machine works (in particular it strongly wants a C style stack and does manual stack-pointer manipulation). I actually got fairly far into the bootstrap process under emscripten, but the minimal lisp interpreter it compiles generated bizarre errors.
I disagree. Clasp could do this because it compiles Common Lisp to LLVM-IR bitcode files (using COMPILE-FILE) as well as directly to native code using LLVM's MCJIT engine (using COMPILE). emscripten (https://github.com/kripken/emscripten) says that it compiles LLVM-IR to JavaScript. I haven't used emscripten, but I believe everything I read on the internet :-) and thus Common Lisp --[Clasp]--> LLVM-IR bitcode files --[emscripten]--> run within browsers.
3 replies →
Sorry to go offtopic but since the you mentioned emscripten I though I'd ask.
Can you explain what emscripten is used for? I'm fully aware of what it does, just not why someone would want to use it. How do you use things like jQuery or access window or document from C and then compile it to js? Or is emscripten specifically for js that doesn't interact with the DOM?
2 replies →
> Common Lisp in the browser, here we come!
You have Parenscript for that: http://common-lisp.net/project/parenscript/
(I haven't used it yer though.)
It works really well. It's a subset of CL that it can compile though so you'll miss out on CLOS, restarts, etc. However the subset does include macros so it's serviceable.
2 replies →
I don't see why we couldn't do this. emscripten (https://github.com/kripken/emscripten) says it takes LLVM-IR to JavaScript. The Clasp (Common Lisp) function COMPILE-FILE compiles to LLVM-IR bitcode files. If you have some time and you would like to do this - hit me up with an email - I'd love to make this happen.
You'll probably run into all the same problems other Common Lisps run in to when they try to shake out the compiler from their runtime, namely that often you really do want to create and run new code at runtime (i.e. EVAL but much more often (compile nil '(lambda () ...)). A common example in CL is creating CLOS dispatch functions with specialized parameters to speed up generic dispatch, which I believe most CLOS implementations do at runtime (certainly SBCL/CMUCL's PCL does). You'll wind up needing to include your compiler, Emscripten itself, and all of their dependencies (LLVM libraries?) which will be quite a huge pile of JavaScript. The alternative is to include a CL interpreter but that has obvious speed problems.
ClojureScript already exists, and even if it's not Common Lisp, its still Lisp with homoiconity, macros and what not.
Ok. Anyone can explain what do I get the downvotes for?
5 replies →
https://github.com/davazp/jscl
Exciting work. CL on the LLVM has been needed.
Is this project something that you're doing to support your own research goals in computational chemistry? If so, can you share how this project fits into the bigger picture?
Finger crossed then that Azule can get a proper moving GC to work with LLVM then, until then a CL on LLVM (even Julia) are stuck with kinda bad GC.
I wonder why start from scratch for an LLVM backend, can't SBCL be used to generate LLVM code?
SBCL doesn't generate LLVM code. My primary goal was Common Lisp with C++ interoperation. It seemed easier at the time to start from the ECL Common Lisp code base and write a new C++ core and a Common Lisp interpreter that always interoperated with C++. As I wrote the compiler and expanded the system I maintained C++ interoperation all the time. There were a hundred problems that I had to discover and solve along the way to maintain a Common Lisp that interoperated with C++. You can get some level of interoperation between ECL and C++ up in a weekend but it won't work with C++ exceptions and RAII and there are dozens of other problems. In retrospect I don't think I would have gotten here starting from ECL because I never really understood the ECL code.
It wasn't from scratch, it was from ECL. And did you mean Azul?
Is the compacting garbage collector mentioned in the article not a proper moving GC? I am not familiar with it.
Yes, the Memory Pool System by Ravenbrook (https://www.ravenbrook.com/project/mps/) is a proper, moving garbage collector. It uses precise GC on the heap and conservative GC on the stack, as does the garbage collector in Steel Bank Common Lisp on x86 chips. I need it because I need my code to run on 100,000 CPU supercomputers with a controlled memory footprint to develop organic nano machines (seriously).
2 replies →
I would point out that MOCL https://wukix.com/mocl pretty much provides this functionality only with full Common Lisp compatibility. It compiles the CL to C and adds methods to expose your functions to Obj-C / C code (if you want). Works just fine. Builds with Xcode, which as I understand, does everything on/via LLVM.
mocl is different from many other implementations: it is a batch compiler for whole programs to C. The generated code does contain a limited evaluator and no compiler. Some dynamic features are gone...
MOCL specifically does not have full CL compatibility.
MOCL is garbage, commercial garbage at that. No thanks.
> Clasp exposes the Clang AST library and the Clang ASTMatcher library. This allows programmers to write tools in Common Lisp that automatically analyze and refactor C++ programs. Clasp can be used to automatically clean-up and refactor large C++ codebases!
Niiice. I want to program C++, but not in C++.
Yes, I'll post more on this and provide example code in the coming weeks on the blog. It's really exciting and it's something that Clasp can do right now even though the Clasp compiler isn't a highly optimizing one yet. Google uses automated Clang refactoring to clean up the googleplex (100 megaLOC). Clasp puts those capabilities into everyones hands now.
Wow, this news almost makes me drool. I will keep my eye on that. I'm on SBCL right now, but I can't wait to see what Lisp can do with LLVM...
I hope it turns out a good implementation since there is a lot of C++ code out there and I do not know C++. But I can comprehend Lisp.
Maybe not-so-related, but "clasp" is also the name of a pretty popular Answer Set Programming (ASP) solver (link http://www.cs.uni-potsdam.de/clasp).
The name is already taken unfortunately: http://www.cs.uni-potsdam.de/clasp/
"Common Lisp Analytical Statistical Package" = CLASP, ca. 1990
http://www.cs.cmu.edu/afs/cs/Web/Groups/AI/lang/lisp/code/ma...
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.138.... http://dl.acm.org/citation.cfm?id=896952
Later another CLASP: Common LISP as simulation program (CLASP)
https://github.com/openlisp/clasp
It was also already taken by a device for fastening things. And my name was already taken by at least three people before me. Somehow we all get along.
Package names in distributions have to be unique though, so this one will probably have to be called clasp-llvm or something like that.
All names are already taken, that's why Wikipedia has the disambiguation pages.
Someone has been building Ocaml's opam on Arch I see.