It now removes empty sets of (relevant) background knowledge to avoid needlessly creating lots of problems (thousands!) when it only needs one, and also stopped it checking array usage when there's at most one written-to/read-from index.
Fixes#90
There was a bug where things scoped in via pragmas were never scoped out again, which was screwing up the local names stack. I then realised/decided that pragmas were really specifications, and decided to put them there in the parser.
The rest of this patch is just some rewiring to allow the special name munging involved in pragmas (they have already got a munged version of their name) and to stop the scoped in pragmas appearing in the AST.
This is necessary because occam files often #INCLUDE something multiple times (e.g. cgtests) and want different names, but this is not the case for #USE.
Now that we support separate compilation, some of the stack sizes for PROCs depend on the stack sizes of other PROCs that were compiled in different files. We can't just assume the default 512 bytes for these "foreign" PROCs, because that often won't be enough. So instead, we must make the stack sizes for the current PROCs depend on (i.e. use in the calculation) the stack sizes of the foreign PROCs.
This dependence adds some issues though. We cannot declare in one C file a const int that depends on the value of an extern const int from another C file (not valid C, it seems). So instead, we move all the stack size declarations to header files, and use #includes and the preprocessor to make sure that the stack sizes are statically determined.
This in turn simplifies the build process in some ways. These headers only need to be compiled by the .occ file that has the main process, by including them all into a C file and compiling that as before. It means that each .occ file only has one .o file resulting (plus two C headers*, and a .inc file) so linking is a bit less confusing.
* I am keeping the two C headers for now, rather than appending the sizes one to the normal header, because I'm not entirely sure whether having one header that the C file depends on may trigger a recompilation that we don't want in some build systems. I can always merge them later if that's not a valid worry.
This just about works, but it allocates the usual 512 bytes for all external PROCs being called, rather than using the actual stack_size for those external PROCs (this causes popcorn to overflow the stack if left with the default 512 bytes)
The second part of the patch is essential, given the first. Otherwise names in different pragmas in the same file can overlap -- this already happened in oak!
I must admit, this was mainly done to allow munged names back in again as valid identifiers.
OEP 144 suggests replacing dot with underscore; this change just allows underscore alongside dot. It won't break any existing code, and seems like something we want anyway, so I think it's a valid thing to do.
One change, based on Adam's suggestion, was to rename the pragma to TOCKEXTERNAL.
Another, also based on Adam's suggestion, was to generate both the munged name and the original name, which allows (along with a previous patch) different files to declare the same PROC, and will remove the need for the occam_ prefix in the backend.
I also stopped using specific states in the lexer, in favour of just using the normal lexing function (which has had its type generalised slightly).
This will allow (along with a few patches in a minute) different occam files to declare the same PROC, and have it resolved correctly based on the order of their declaration, just like if it was all in one file.
The solution is a bit hacky, but this was an important problem. If your PRAGMA failed to parse, that was worthy of a warning. But if that then caused the parse to fail, all you would get is the parser error (could not find name), and you would never see the warnings about the pragmas not being recognised. So now the pragmas are shoved into the error (using a basic encoding) and pulled out and issued if the parser dies.
The separately compiled occam PROCs now use #PRAGMA OCCAMEXTERNAL, which also discards the "= number" thing at the end. These PROCs then need to be processed differently when adding on the sizes (C externals have one size per dimension, occam externals have the normal array of sizes).
We also now record which processes were originally at the top-level, and keep their original names (i.e. minus the _u43 suffixes) plus an "occam_" prefix to avoid collisions.
