Skip to content

Function I found useful - blockwise indexed task submission #35

Description

@UH-Comet-Group

I found this function useful - sort of a blockwise pdotimes. One could do the same with map, but
this turned out to be simpler to use for me. If I'm not missing something, it might be a useful standard functionality (after changing the awful name).

I think it's not quite the same as pdotimes, because pdotimes calls its tasks by singles,
so that any overheads are repeated. Below, the task given by func can have expensive overheads.

;; utility function that takes a task indexed i1..i2, and
;; calls (func j1 j2) in parallel to span ranges of i1,i2

(defun chop-up-indexed-tasks-into-threads (i1 i2 func &key (nthreads nil))
  (loop with ncpu = (or nthreads (lparallel:kernel-worker-count))
	with ntot = (- i2 i1)
	with nblock = (ceiling ntot ncpu) ;; number in a block
	with ithread = 0
	with channel = (lparallel:make-channel)
	for j1 = i1 then (+ 1 j1 nblock)
	for j2 = (min (+ j1 nblock) i2)
	do (lparallel:submit-task channel func j1 j2)
	   (incf ithread)
	until (= j2 i2)
	finally
	   (loop for k below ithread
		 do (lparallel:receive-result channel))))

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions