Discussion:
Can't M-x compile-defun `edebug' because dynamic variables are falsely taken as lexical.
(too old to reply)
Alan Mackenzie
2017-01-03 14:14:44 UTC
Permalink
Hello, Emacs.

I'm in Emacs 25.1.

I want to M-x compile-defun the function `edebug'. When I attempt this,
I get the "warning" messages:

Warning: Unused lexical variable `edebug-break'
Warning: Unused lexical variable `edebug-global-break'
Warning: Unused lexical variable `edebug-break-condition'
Warning: Unused lexical variable `edebug-break-data'
Warning: Unused lexical variable `edebug-breakpoints'

. These variables are dynamic ones, declared earlier on in the file
like this:

(defvar edebug-breakpoints)

, i.e. without initialising expressions. I tried C-x C-e on each such
declaration, then repeating the compile-defun, to no avail.
lexical-binding is set in edebug.el.

Emacs should be able to compile successfully in such circumstances,
surely?.

What am I doing wrong, here?
--
Alan Mackenzie (Nuremberg, Germany).
Stefan Monnier
2017-01-03 18:35:19 UTC
Permalink
Post by Alan Mackenzie
I want to M-x compile-defun the function `edebug'.
Looks like `compile-defun` is another function that needs to be changed
to use `eval-sexp-add-defvars`.


Stefan
Alan Mackenzie
2017-01-03 21:32:28 UTC
Permalink
Hello, Stefan.
Post by Stefan Monnier
Post by Alan Mackenzie
I want to M-x compile-defun the function `edebug'.
Looks like `compile-defun` is another function that needs to be changed
to use `eval-sexp-add-defvars`.
It looks like you've already done that, possibly on 2015-10-29. Looking
at the source (now I know what to look for), eval-sexp-add-defvars is
also in Emacs 25.1's source for compile-defun. So, something else has
gone wrong.....

I think the something else might be in Fdefvar. At its outermost level
Fdefvar contains:

###############################################
(Lisp_Object args)
{
Lisp_Object sym, tem, tail;

sym = XCAR (args);
tail = XCDR (args);

if (CONSP (tail))
{
......
/* Do it before evaluating the initial value, for self-references. */
XSYMBOL (sym)->declared_special = 1;
......
}
###############################################

, so the question becomes why is setting the declared_special flag done
inside the "if (CONSP (tail))" rather than outside? I.e. why is it only
done when a variable has an initialisation?
Post by Stefan Monnier
Stefan
--
Alan Mackenzie (Nuremberg, Germany).
Stefan Monnier
2017-01-03 21:48:15 UTC
Permalink
Post by Alan Mackenzie
It looks like you've already done that, possibly on 2015-10-29. Looking
at the source (now I know what to look for), eval-sexp-add-defvars is
also in Emacs 25.1's source for compile-defun.
Oh, indeed. Not sure why you're seeing what you're seeing, then.
At least on Emacs's master branch I can't seem to reproduce your problem
(don't have a fresh emacs-25 build to try it right now).
Post by Alan Mackenzie
, so the question becomes why is setting the declared_special flag done
inside the "if (CONSP (tail))" rather than outside? I.e. why is it only
done when a variable has an initialisation?
Because this code is only relevant when you evaluate (defvar <foo>),
whereas here we're not evaluating it, we're only processing it for the
byte-compiler.

There's also the important difference that a (defvar <foo>) only
has effect for the code in the same file rather than having a global
effect, so you can do (defvar toto) and then use `toto` as
a dynamically-scoped variable in your file without wreaking havoc in all
other files which happen to also use `toto` as a (lexical) variable.


Stefan
Alan Mackenzie
2017-01-04 13:39:48 UTC
Permalink
Hello, Stefan.
Post by Stefan Monnier
Post by Alan Mackenzie
It looks like you've already done that, possibly on 2015-10-29. Looking
at the source (now I know what to look for), eval-sexp-add-defvars is
also in Emacs 25.1's source for compile-defun.
Oh, indeed. Not sure why you're seeing what you're seeing, then.
At least on Emacs's master branch I can't seem to reproduce your problem
(don't have a fresh emacs-25 build to try it right now).
For what it's worth, I can't reproduce the problem any more. I must
have got my Emacs 25.1 into a strange state, somehow.
Post by Stefan Monnier
Post by Alan Mackenzie
, so the question becomes why is setting the declared_special flag done
inside the "if (CONSP (tail))" rather than outside? I.e. why is it only
done when a variable has an initialisation?
Because this code is only relevant when you evaluate (defvar <foo>),
whereas here we're not evaluating it, we're only processing it for the
byte-compiler.
Understood. Thanks.
Post by Stefan Monnier
There's also the important difference that a (defvar <foo>) only
has effect for the code in the same file rather than having a global
effect, so you can do (defvar toto) and then use `toto` as
a dynamically-scoped variable in your file without wreaking havoc in all
other files which happen to also use `toto` as a (lexical) variable.
Ouch! There seems to be a clash between symbols (which are global,
dynamic) and lexical variables (which are local and "invisible"): the
only way to mark a variable as lexical is at the global level.

However, I'm seeing something else strange (still on 25.1). I do this:

(i) M-x load-library edebug.
(ii) visit edebug.el.
(iii) instrument `edebug' for edebugging (yes, I know ;-).
(iv) M-: (edebug).
(v) step through the function to just after the let bindings.
(vi) e (special-variable-p 'edebug-breakpoints).

This last returns nil. This suggests edebug-breakpoints has been bound
as a lexical variable, rather than a dynamic one. (There is a defvar
for it earlier in the file.) This is surely not right.

What is going on here? Is it, perhaps, a bug?

However, edebug.elc still seems to work, though I haven't successfully
mananged to run `edebug' recently.
Post by Stefan Monnier
Stefan
--
Alan Mackenzie (Nuremberg, Germany).
Stefan Monnier
2017-01-04 15:23:16 UTC
Permalink
Post by Alan Mackenzie
Ouch! There seems to be a clash between symbols (which are global,
dynamic) and lexical variables (which are local and "invisible"): the
only way to mark a variable as lexical is at the global level.
Yes, there are some subtleties there. The main issue is that every
occurrence of an identifier, whether a binding occurrence or a "use" can
be lexical or dynamic, but we don't want the coders to have to
constantly specify which uses and which bindings are lexical and which
are dynamic. So we offer ways to "mark" symbols are "this one uses
dynamic scoping". There are 2 such ways:
- *Evaluation* of (defvar VAR VAL . REST) marks this variable as being
dynamically scoped in all the code that will be compiled in this session.
- *Compilation* of (defvar VAR . REST) marks this variable as being
dynamically scoped in the current compilation unit.
Post by Alan Mackenzie
(vi) e (special-variable-p 'edebug-breakpoints).
This last returns nil. This suggests edebug-breakpoints has been bound
as a lexical variable, rather than a dynamic one. (There is a defvar
for it earlier in the file.) This is surely not right.
special-variable-p only indicates if (defvar VAR VAL . REST) was evaluated.


Stefan
Alan Mackenzie
2017-01-04 20:04:58 UTC
Permalink
Hello, Stefan
Post by Stefan Monnier
Post by Alan Mackenzie
Ouch! There seems to be a clash between symbols (which are global,
dynamic) and lexical variables (which are local and "invisible"): the
only way to mark a variable as lexical is at the global level.
Yes, there are some subtleties there. The main issue is that every
occurrence of an identifier, whether a binding occurrence or a "use" can
be lexical or dynamic, but we don't want the coders to have to
constantly specify which uses and which bindings are lexical and which
are dynamic. So we offer ways to "mark" symbols are "this one uses
- *Evaluation* of (defvar VAR VAL . REST) marks this variable as being
dynamically scoped in all the code that will be compiled in this session.
- *Compilation* of (defvar VAR . REST) marks this variable as being
dynamically scoped in the current compilation unit.
Post by Alan Mackenzie
(vi) e (special-variable-p 'edebug-breakpoints).
This last returns nil. This suggests edebug-breakpoints has been bound
as a lexical variable, rather than a dynamic one. (There is a defvar
for it earlier in the file.) This is surely not right.
special-variable-p only indicates if (defvar VAR VAL . REST) was evaluated.
So it would seem. There is a bug in the elisp manual, which says that a
variable being declared by defvar will cause special-variable-p to
return t for it. The doc string looks right, though far from helpful
for anybody who doesn't already know variable binding inside out, and
even a bit cryptic for those who do.

special-variable-p would appear to be a near useless function, since it
doesn't do what it's name says.

"Defining Variables" in the elisp manual states that "(defvar foo)" makes
foo a special variable; yet "(special-variable-p 'foo)" returns nil. This
has got to be a bug, of some sort.

There appears to be no way of checking whether a variable's binding is
(or will be) lexical.

I can foresee quite a bit of confusion happening over all this, if it
hasn't happened already.

Am I missing something, or is this all really as incoherent as it
appears to me?
Post by Stefan Monnier
Stefan
--
Alan Mackenzie (Nuremberg, Germany).
Stefan Monnier
2017-01-04 21:49:15 UTC
Permalink
Post by Alan Mackenzie
special-variable-p would appear to be a near useless function, since it
doesn't do what it's name says.
It's useful for the compiler, but it's mostly internal, indeed.
Post by Alan Mackenzie
There appears to be no way of checking whether a variable's binding is
(or will be) lexical.
In which context do you need/want to do that (I ask because how to do
it (and even if it can be done) depends on the details)?


Stefan
Alan Mackenzie
2017-01-04 22:02:43 UTC
Permalink
Hello, Stefan.
Post by Stefan Monnier
Post by Alan Mackenzie
special-variable-p would appear to be a near useless function, since it
doesn't do what it's name says.
It's useful for the compiler, but it's mostly internal, indeed.
OK.
Post by Stefan Monnier
Post by Alan Mackenzie
There appears to be no way of checking whether a variable's binding is
(or will be) lexical.
In which context do you need/want to do that (I ask because how to do
it (and even if it can be done) depends on the details)?
I honestly don't know. I presume that there will be code (other than
the byte compiler) which will want to make a distinction. Maybe there's
not. After (defvar foo), the byte compiler seems able to handle foo as
a dynamic variable. I'm still trying to figure out how, given that that
flag denoting a special variable in the struct symbol doesn't get set.

I'm just thoroughly confused about the whole business of whether (defvar
foo) creates a special variable or not. At the moment, I'm not even
quite sure any more exactly what a special variable is. I knew this
morning. :-(
Post by Stefan Monnier
Stefan
--
Alan Mackenzie (Nuremberg, Germany).
Stefan Monnier
2017-01-04 22:26:53 UTC
Permalink
Post by Alan Mackenzie
Post by Stefan Monnier
In which context do you need/want to do that (I ask because how to do
it (and even if it can be done) depends on the details)?
I honestly don't know. I presume that there will be code (other than
the byte compiler) which will want to make a distinction.
There might, indeed, but it's very rare: in most cases rather than
determining which kind of binding will happen, you want to decide/impose
which binding will happen.

I'm not sure how Common-Lip handles it, but AFAICT there is no
equivalent to special-variable-p there, so they don't seem to offer
a way to find out whether a binding will be lexical or dynamic.
Post by Alan Mackenzie
After (defvar foo), the byte compiler seems able to handle foo as
a dynamic variable. I'm still trying to figure out how,
The byte-compiler *sees* the defvar, which lets it keep a note
internally (in a data-structure which keeps track of the current
context, which also includes information about which vars that are
let-bound in the surrounding code were bound lexically, so as to know
when we see a reference to var `foo' whether we should look for `foo' in
the dynamic context or in the lexical context (and if so, where in that
context)).


Stefan
Drew Adams
2017-01-04 22:44:23 UTC
Permalink
I _almost_ wanted to offer this, as a way to check dynamic vs lexical:

`symbol-value', wrapping with `condition-case', checking for
`void-variable' error and returning `nil' in that case.

But if there are a dynamic var and a lexical var with the same name,
`symbol-value' just sees the former - no real way to test the latter.
Alan Mackenzie
2017-01-05 10:54:27 UTC
Permalink
Hello, Stefan.
Post by Stefan Monnier
Post by Alan Mackenzie
Post by Stefan Monnier
In which context do you need/want to do that (I ask because how to do
it (and even if it can be done) depends on the details)?
I honestly don't know. I presume that there will be code (other than
the byte compiler) which will want to make a distinction.
There might, indeed, but it's very rare: in most cases rather than
determining which kind of binding will happen, you want to decide/impose
which binding will happen.
OK.
Post by Stefan Monnier
I'm not sure how Common-Lip handles it, but AFAICT there is no
equivalent to special-variable-p there, so they don't seem to offer
a way to find out whether a binding will be lexical or dynamic.
Post by Alan Mackenzie
After (defvar foo), the byte compiler seems able to handle foo as
a dynamic variable. I'm still trying to figure out how,
The byte-compiler *sees* the defvar, which lets it keep a note
internally (in a data-structure which keeps track of the current
context, which also includes information about which vars that are
let-bound in the surrounding code were bound lexically, so as to know
when we see a reference to var `foo' whether we should look for `foo' in
the dynamic context or in the lexical context (and if so, where in that
context)).
Thanks, I'll have a look at that sometime.
Post by Stefan Monnier
Stefan
--
Alan Mackenzie (Nuremberg, Germany).
Continue reading on narkive:
Loading...