Simpler FFI in Owl Lisp

This article explains in detail how to use owl-libffi.

owl-libffi is a simple, lightweight libffi abstraction layer. It allows for opening shared libraries and calling foreign functions in a lispy style.

The steps for calling a foreign function are as follows:

Basic values

Simple, right? Let's see how one can accomplish that from zero to lots of syntax sugar. The following 2 examples will:

(import
 (owl toplevel)
 (ffi))

(λ (_)
  (let* ((open-library "libc.so")
         (malloc-proto (prep-cif (list int32) pointer))
         (strcpy-proto (prep-cif (list pointer pointer) pointer))
         (write-proto  (prep-cif (list int32 pointer uint32) uint32))
         (free-proto   (prep-cif (list pointer) void))
         (malloc-sym   (dlsym lib "malloc"))
         (strcpy-sym   (dlsym lib "strcpy"))
         (write-sym    (dlsym lib "write"))
         (free-sym     (dlsym lib "free")))
    (let ((memptr (call malloc-proto malloc-sym '(32)))
          (str "Hello, World!\n"))
      (call strcpy-proto strcpy-sym (list memptr str))
      (let ((bytes-written (call write-proto write-sym (list stdout memptr (string-length str)))))
        (print "write(2) wrote " bytes-written " bytes to stdout"))
      (call free-proto free-sym (list memptr))))
  0)
$ ol -x c -o test.c test.scm
$ cc -I/tmp/owl/c -DPRIM_FP_API -DPRIM_CUSTOM test.c ffi.c `pkg-config --cflags --libs libffi` -ldl -o test
$ ./test
Hello, World!
write(2) wrote 14 bytes to stdout

Hmmm, this works, but it's not very pleasant to write. How about sprinkling some pre-defined macros on top?

(import
 (owl toplevel)
 (ffi))

(λ (_)
  (lets ((malloc strcpy write free
          (funcs (open-library "libc.so")
            ("malloc" pointer int32)
            ;; symbol ret-val arg1 ... argN
            ("strcpy" pointer pointer pointer)
            ("write"  int32 int32 pointer uint32)
            ("free"   void pointer))))
    (let ((memptr (malloc 32))
          (str "Hello, World!\n"))
      (strcpy memptr str)
      (let ((bytes-written (write stdout str (string-length str))))
        (print "write(2) wrote " bytes-written " bytes"))
      (free memptr)))
  0)

After compiling it in the same way, it yields the same result, but look how much nicer it is to write and use! Notice how we're not call-ing the functions in a special way - they're already wrapped in lisp lambdas for us, so we can use them like normal functions.

Using some sophisticated owl magic, we can also inspect that the Hello, World! string is - in fact - stored at address memptr.

(import
 (owl toplevel)
 (ffi)
 (prefix (owl sys) sys/))

(define (u8ptr->number bvec)
  (fold (λ (a b) (bior (<< a 8) b)) 0 (reverse (bytevector->list bvec))))
                                    ;; ^^^^^^^ are you lsb or msb

(λ (_)
  (lets ((malloc strcpy free
          (funcs (open-library "libc.so")
            ("malloc" pointer int32)
            ("strcpy" pointer pointer pointer)
            ("free"   void pointer))))
    (let ((memptr (malloc 32))
          (str "Hello, World!\n"))
      (strcpy memptr str)
      (format stdout "memptr: bvec=~s, ptr=0x~x\n" (str* memptr) (u8ptr->number memptr))
      (format stdout "str: ~s" (sys/mem-string (u8ptr->number memptr)))
      ;;                            ^^^^^^^^^^
      ;; mem-string accepts the pointer as a number, so we need to convert it from our
      ;; bytevector representation
      (free memptr)))
  0)

After compiling and running, it yields

memptr: bvec="#u8(48 180 157 191 197 85 0 0)", ptr=0x55c5bf9db430
str: "Hello, World!\n"

Structs

To create a structure, use a defstruct macro, which returns (values Tstruct make-struct), where Tstruct is a type description that can be later used as in a function signature, and make-struct is a function that creates a struct from passed arguments.

The following code shows how this can be used.

;; struct tm {
;;   int         tm_sec;    /* Seconds          [0, 60] */
;;   int         tm_min;    /* Minutes          [0, 59] */
;;   int         tm_hour;   /* Hour             [0, 23] */
;;   int         tm_mday;   /* Day of the month [1, 31] */
;;   int         tm_mon;    /* Month            [0, 11]  (January = 0) */
;;   int         tm_year;   /* Year minus 1900 */
;;   int         tm_wday;   /* Day of the week  [0, 6]   (Sunday = 0) */
;;   int         tm_yday;   /* Day of the year  [0, 365] (Jan/01 = 0) */
;;   int         tm_isdst;  /* Daylight savings flag */
;;
;;   long        tm_gmtoff; /* Seconds East of UTC */
;;   const char *tm_zone;   /* Timezone abbreviation */
;; };

(import
 (owl toplevel)
 (prefix (owl sys) sys/)
 (ffi))

(define (u8ptr->number bvec)
  (fold (λ (a b) (bior (<< a 8) b)) 0 (reverse (bytevector->list bvec))))

(λ (_)
  (lets ((Ttm make-tm (defstruct int32 int32 int32 int32 int32 int32 int32 int32 int32 int64 pointer))
         ;; see above for struct tm { ... } declaration
         (malloc strcpy free strftime
          (funcs (open-library "libc.so")
            ("malloc"   pointer int32)
            ("strcpy"   pointer pointer pointer)
            ("free"     void pointer)
            ("strftime" uint32 pointer uint32 pointer pointer))))
            ;;                                        ^^^^^^^
            ;; strftime requires a tm*, so instead of Ttm, we set the function prototype to a plain pointer
    (let ((mem (malloc 128)))
      (let-structs ((tm (make-tm 0 37 21 2 3 105 7 92 0 (* 60 60 2) "CEST")))
        ;; let-structs is a let-like macro that defers `free-struct` for every struct created
        (strftime mem 128 "%Y-%m-%d %H:%M" tm))
      (print (sys/mem-string (u8ptr->number mem)))
      (free mem)))
  0)

This yields:

2005-04-02 21:37

This could - of course - also be accomplished using builtin owl libraries, but this example is meant to show how one could use structs.

availability

This library should work on linux/unix. An MS Windows port for owl-winrt is planned.

resources