Closure Fun 1

Posted by Aaron Feng Mon, 30 Jun 2008 23:52:00 GMT

PHP has a string tokenizing function called strtok that has an interesting interface compared to many languages. What makes strtok function interesting is that creating the tokenizer and generating tokens utilize the same function. For example:

$tok = strtok("closure fun", "  ");

while ($tok !== false) {
    echo "$tok\n";
    $tok = strtok("  ");
}

The code above tokenizes the input string by a space, and outputs each token on a newline. What makes this function interesting is the ability to maintain state! I don't know what happens under the hood, but it appears to "remember" the input string after the tokenizer has been created. This behavior can be easily mimicked by using a closure variable. Below is an implementation in lisp:

(let ((rest nil))
  (defun strtok (string &optional (token nil token-supplied-p))
    (check-type string string)
    (when token-supplied-p (check-type token string))

    (when (and string token) (setf rest string))
    (let ((items nil)
          (output nil))
      (loop until (or (null rest)) do
           (setf items (cl-ppcre:split 
                        (funcall (lambda () (if token token string)))
                        rest 
                        :limit 2)) 
           (setf output (first items))
           (setf rest (second items))

           (when (not (string= output "")) 
             (return-from strtok output))))))

The "rest" variable outside of the function declaration is the closure variable. The "rest" is in fact invisible to the outside world, and it allows the function to maintain state!

Comments

Leave a response

  1. Leslie P. Polzer Thu, 03 Jul 2008 09:49:49 GMT

    This generator pattern is very common in C, and PHP borrows a lot from C (see strtok(3) for the C interface of this function). Python guys like to do stuff like this, too.

    How it works: the binding to REST is established on execution of the code once (and initialized to NIL). After that, the content of the variable is maintained by the code that is executed in the function body.

    Note that this implementation of strtok isn't thread-safe. Multiple threads will share REST among them.

    By the way, you can also just write

    (let (rest) ...

    if you want.

Comments