Comment by Joker_vD
11 hours ago
> I can directly inspect the result of the preprocessor makes macros easier to debug
Tangential note but: why does this require a separate re-invocation of the compiler with the '-E' flag? Say I have this silly example with a comma instead of the semicolon inside W() macro:
#define W(a, b) b, int a
#define X(a, b) W(b, a)
int kekw(int x) {
int z = X(23, k) = x;
return z + k;
}
When I shove it into GCC without -E, it says
<source>: In function 'int kekw(int)':
<source>:1:21: error: expected unqualified-id before 'int'
1 | #define W(a, b) b, int a
| ^~~
<source>:2:18: note: in expansion of macro 'W'
2 | #define X(a, b) W(b, a)
| ^
<source>:5:13: note: in expansion of macro 'X'
5 | int z = X(23, k) = x;
| ^
<source>:6:16: error: 'k' was not declared in this scope
6 | return z + k;
|
instead of, you know, showing me the expanded text so that I could see what's actually wrong with the generated variable declarations? Something like
<source>: In function 'int kekw(int)':
<source>:1:21: error: expected unqualified-id before 'int'
1 | #define W(a, b) b, int a
| ^~~
<source>:2:18: note: in expansion of macro 'W'
2 | #define X(a, b) W(b, a)
| ^
<source>:5:13: note: in expansion of macro 'X'
5 | int z = X(23, k) = x;
| ^
<source>:5:-: note: the expanded source text is
| int z = 23, int k = x;
| ^~~
<source>:6:16: error: 'k' was not declared in this scope
6 | return z + k;
|
perhaps? I understand that tracking the lines/columns inside the expanded text is highly non-trivial (although collapsing them all onto the position of the top-level macro is already just as non-trivial), but still. Show me what the source that you actually can't compile, not the three-steps removed precursor to it — I sure as hell can't expand macros in my head.
Also, what is this first error message about in the first place? Why the hell does it expect unqualified-id before "int"? If you put "int" after the unqualified-id, it still would be illegal; and an unqualified-id is not the only thing that can follow the comma in this context anyway.
During translation of source text to a C program, the source text goes through several phases. It turns raw bytes into preprocessor tokens, then preprocessor tokens get expanded into more preprocessor tokens and then preprocessor tokens get translated into C tokens, which are fed into the C parser. There is no “expanded text”, only expanded tokens. Generally this is implemented as a state machine that produces tokens on demand, so there is no expanded text. When GCC shows you that error message, it uses the recorded source location and looks it up in the original source file. This is trivial to do. Expanding just one line of the source file would either require re-invoking the preprocessor from the beginning or storing already consumed preprocessor tokens.
Obviously this is possible, but no one has gotten around to implementing this feature and it’s not clear you would want either solution: buffering preprocessor tokens in case you need to expand a line for an error message greatly increases memory usage and the vast majority of compilations are successful ones with no errors, slowing down the normal case. And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
> There is no “expanded text”, only expanded tokens.
If the compiler supports "-E", there absolutely is some equivalent of the expanded text, with (somewhat limited) whitespace info available: if you insert whitespace into between "b" and the comma in "b,", then "-E" invocation will dutifully show it. Yes, multiple spaces are collapsed into one, but their presence/absence is tracked already.
> Expanding just one line of the source file would either require re-invoking the preprocessor from the beginning or storing already consumed preprocessor tokens.
Yeah, keep the whole expansion of the latest top-level (not nested) macro. I believe some bytecode-like compression scheme (a replay of that state machine you mentioned) à la LZ would work fairly well, with minor additional memory necessary. Hell, scratch that: since #define/#undef are impossible inside a macro, you don't even need to remember the expansion of the macro, you simply need to keep the current macro definitions and remember what was the last macro you tried to expand — if your error happened inside its expansion, just re-expand again. Zero additional memory needed, since this information is already stored for the current way of error reporting — you can see in the current output that it remembers what macros it expanded and where, the whole path down to the offending token: at 1:21 "in expansion of macro 'W'" which was at 2:18 "in expansion of macro 'X'" which was at 5:13.
> And re-invoking the preprocessor to show you one line on every error message makes error reporting slow.
Eh, it's on the whole faster than making the programmer re-run with "-E" option manually.
> If the compiler supports "-E", there absolutely is some equivalent of the expanded text, with (somewhat limited) whitespace info available: if you insert whitespace into between "b" and the comma in "b,", then "-E" invocation will dutifully show it. Yes, multiple spaces are collapsed into one, but their presence/absence is tracked already.
Whitespace is significant in the preprocessor so of course it tracks whitespace. Whitespace isn’t dropped until phase 7 when pp-tokens are converted to C tokens.
As I said it’s not impossible, write the patch and see if GCC accepts it.
1 reply →