Threads and Parallelism

41.1 bordeaux-threads (Portable Threads)

Special variables and threads: global values are shared, but dynamic bindings (let of a special variable) are thread-local. Pass :initial-bindings or bind inside the thread function as needed.

SBCL specifics: sb-thread, atomic operations (sb-ext:atomic-incf on fixnum struct slots/arrays, sb-ext:compare-and-swap), and synchronized hash tables.

(ql:quickload "bordeaux-threads")

(defvar *counter* 0)
(defvar *lock* (bt:make-lock "counter-lock"))

(let ((threads (loop repeat 4
                     collect (bt:make-thread
                              (lambda ()
                                (dotimes (i 10000)
                                  (bt:with-lock-held (*lock*)
                                    (incf *counter*))))
                              :name "worker"))))
  (mapc #'bt:join-thread threads)
  *counter*)                            ; 40000

;; Condition variables
(defvar *queue* '())
(defvar *queue-lock* (bt:make-lock))
(defvar *queue-cv* (bt:make-condition-variable))

(defun enqueue (item)
  (bt:with-lock-held (*queue-lock*)
    (setf *queue* (append *queue* (list item)))
    (bt:condition-notify *queue-cv*)))

(defun dequeue ()
  (bt:with-lock-held (*queue-lock*)
    (loop while (null *queue*) do (bt:condition-wait *queue-cv* *queue-lock*))
    (pop *queue*)))

41.2 lparallel (Task Parallelism)

Other libraries: chanl (CSP channels), calispel, cl-async/blackbird (event loop + promises), lfarm (distributed lparallel).

(ql:quickload "lparallel")
(setf lparallel:*kernel* (lparallel:make-kernel 4))   ; a pool of 4 worker threads

(lparallel:pmap 'list (lambda (n) (* n n)) '(1 2 3 4))          ; (1 4 9 16)
(lparallel:preduce #'+ (loop for i from 1 to 1000000 collect i)) ; 500000500000
(let ((f (lparallel:future (expensive-computation))))
  (do-other-work)
  (lparallel:force f))                                           ; wait for the result
(lparallel:plet ((a (compute-a)) (b (compute-b))) (+ a b))       ; parallel let
(lparallel:pdotimes (i 10) (process i))
;; Channels and promises: lparallel:make-channel, submit-task, receive-result, promise, fulfill
(lparallel:end-kernel :wait t)