Surrounding Lexical Variable Reference in the Body of defun

[Deokhwan Kim]

Hi there,

the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.

It was a great shock to me because it sounded quite awkward and Common Lisp does not have such restriction AFAIK. Rather, I suspected that I might misunderstand what the sentence really meant. So I decided to make some experiments with the following code stored in foo.el:

;;; -*- lexical-binding: t -*-
(let ((x 0))
(defun counter ()
(setq x (1+ x))))

(message "%d" (counter))
(message "%d" (counter))

Surprisingly, when I ran it in the form of source code, it worked:

$ emacs -Q -batch -l foo.el
1
2

On the other hand, when I tried to byte-compile it, I got the following warning messages:

$ emacs -Q -batch -f batch-byte-compile foo.el
In toplevel form:
foo.el:2:1:Warning: Function counter will ignore its context (x)
foo.el:2:1:Warning: Unused lexical variable `x'
foo.el:4:11:Warning: reference to free variable `x'
foo.el:4:17:Warning: assignment to free variable `x'

In end of data:
foo.el:8:1:Warning: the function `counter' is not known to be defined.
Wrote foo.elc

When I ran the resulting byte-compiled code, I got an error as the manual claims:

$ emacs -Q -batch -l foo.elc
Symbol's value as variable is void: x

Now I'm so confused. Here are my two questions:

1. Why does this restriction exists? Is it inevitable because of some design decision of Emacs? Or is it temporary and removed in a (near) future release?
2. Why does the original source code behave differently from its compiled code?

Best regards,
Deokhwan Kim

[Andreas Röhler]

Post by Deokhwan Kim
Hi there,

the code in the body of a defun or defmacro cannot refer to surrounding lexical variables.

;;; -*- lexical-binding: t -*-
(let ((x 0))
(defun counter ()
(setq x (1+ x))))
(message "%d" (counter))
(message "%d" (counter))
$ emacs -Q -batch -l foo.el
1
2
$ emacs -Q -batch -f batch-byte-compile foo.el
foo.el:2:1:Warning: Function counter will ignore its context (x)
foo.el:2:1:Warning: Unused lexical variable `x'
foo.el:4:11:Warning: reference to free variable `x'
foo.el:4:17:Warning: assignment to free variable `x'
foo.el:8:1:Warning: the function `counter' is not known to be defined.
Wrote foo.elc
$ emacs -Q -batch -l foo.elc
Symbol's value as variable is void: x
1. Why does this restriction exists? Is it inevitable because of some design decision of Emacs? Or is it temporary and removed in a (near) future release?
2. Why does the original source code behave differently from its compiled code?
Best regards,
Deokhwan Kim

as this question isn't raised first time, maybe Emacs developers consider to revert that change?

[Stefan Monnier]

Post by Deokhwan Kim
1. Why does this restriction exists? Is it inevitable because of
some design decision of Emacs? Or is it temporary and removed in
a (near) future release?

It is because of a technical limitation in the byte-compiler's handling
of dynamically loaded docstrings.

You can work around this by replacing your defun
with (defalias counter (lambda () (setq x (1+ x)))).

I believe this is fixed in the Emacs trunk (where defun is just a macro
that expands to a defalias+lambda and where the docstrings handling was
improved accordingly).


Stefan

[j***@gmail.com]

Does this mean that if you build from emacs trunk then the code above should compile and run just fine? In which case is the manual wrong (or at least out of date) about warning you not to refer to the surrounding lexical environment from inside a defun?

Justin

Post by Stefan Monnier
I believe this is fixed in the Emacs trunk (where defun is just a macro
that expands to a defalias+lambda and where the docstrings handling was
improved accordingly).
Stefan

[Deokhwan Kim]

For reference, Revision 108427 (especially, lisp/emacs-lisp/byte-run.el) looks like bringing that change:

http://bzr.savannah.gnu.org/lh/emacs/trunk/revision/108427

- Deokhwan Kim

Post by Stefan Monnier
I believe this is fixed in the Emacs trunk (where defun is just a macro
that expands to a defalias+lambda and where the docstrings handling was
improved accordingly).