Commit Graph

55 Commits

Author SHA1 Message Date
Matthew Flatt
b8c1ce63c6 add option to omit RTD descriptions in fasl output
original commit: 294ca9da084d76aa7b649059856066a1f86fe21b
2020-07-14 20:22:59 -06:00
Matthew Flatt
ec05bac0cf add "externals" fasl support, allow non-strings in sfd
"Externals" supports fasling with some values lifted out an provided
separately.

Lifting the restriction on source file descriptor paths, formerly to
strings, means that paths can be represented in a different way, and
they can be fasled through a different means than the built-in
encodings.

original commit: b6b0ae67b08f2e9bc8b7fafe5ebad0375b6ce9db
2020-07-14 20:22:59 -06:00
Matthew Flatt
9e1c89b575 add compile-omit-concatenate-support
Perform slightly less work on every fasl or unfasl of compiled code
that does not need to support `concatenate-object-files`.

original commit: 410985b062a3e2a4cc48da583167254d4adad2d2
2020-07-13 07:38:41 -06:00
Matthew Flatt
fd3b903c1c sync with https://github.com/cisco/ChezScheme on fasl compression
Merge changes in the way that fasl streams are compressed. The new
approach makes compression explicit in the fasl representation, which
means that tricks like uzing zcat on a fasl file will no longer work
(at least not efficiently).

original commit: 167ac7294a2dc400821e4336f0cfc4de621efe97
2020-07-12 19:07:05 -06:00
Matthew Flatt
c4ffe39efb fix leak related to object counts
When collecting to the maximum generation with object counts enabled,
a structure type would effectively become permanently reachable.

Also, add `bytes-finalized` to report how many bytes were associated
with guardian-based finalization by the most recent collection.

original commit: 852f5e2de95a26d3500321c4d4d732407945a57a
2020-04-16 16:16:13 -06:00
dybvig
0a5700cef6 support for internal fasl compression to allow seeking past compile-time info at run time and run-time info at compile time
- the collector now releases bignum temporaries in the collector
  rather than relocating them so we don't keep around huge bignum
  temporaries forever.
     gc.c
- removed the presumably useless vector-handling code from load()
  which used to be required to handle fasl groups.
     scheme.c
- object files are no longer compressed as a whole, and the parameter
  compile-compressed is no longer defined.  instead, the individual
  fasl objects within an object file are compressed whenever the
  new parameter fasl-compressed is set to its default value, #t.
  this allows the fasl reader to seek past portions of an object
  file that are not of interest, i.e., visit-only code and data
  when "revisiting" an object file and revisit-only code and data
  when "visiting" an object file.  the compressed portions are
  compressed using the format and level specified by the compress-format
  and compress-level parameters.  the C-coded fasl reader and
  boot-file loader no longer handle compressed files; these are
  handled, less efficiently, by the Scheme entry point (fasl-read).
  a warning exception is raised the first time a program attempts
  to create or read a compressed fasl file.
    7.ss, s/Mf-base, back.ss, bytevector.ss, cmacros.ss, compile.ss,
    fasl-helpers.ss, fasl.ss, primdata.ss, strip.ss, syntax.ss,
    externs.h, fasl.c, gc.c, scheme.c, thread.c,
    mats/6.ms, mats/7.ms, mats/bytevector.ms, mats/misc.ms, patch*,
    root-experr*,
    intro.stex, use.stex, io.stex, system.stex,
    release_notes.stex
- added begin wrappers around many of the Scheme source files that
  contained multiple expressions to cut down the number of top-level
  fasl objects and increase compressibility.  also removed the
  string filenames for debugging at the start of each file that had
  one---these are best inserted universally by a modified compile-file
  during a debugging session when desired.  also removed unnecessary
  top-level placeholder definitions for the assignments that follow.
    4.ss, 5_1.ss, 5_2.ss, 5_3.ss, 5_7.ss, 6.ss, 7.ss, bytevector.ss,
    cafe.ss, cback.ss, compile.ss, cp0.ss, cpcommonize.ss, cpletrec.ss,
    cpnanopass.ss, cprep.ss, cpvalid.ss, date.ss, engine.ss, enum.ss,
    env.ss, event.ss, exceptions.ss, expeditor.ss, fasl.ss, foreign.ss,
    format.ss, front.ss, ftype.ss, inspect.ss, interpret.ss, io.ss,
    library.ss, mathprims.ss, newhash.ss, pdhtml.ss, pretty.ss,
    prims.ss, primvars.ss, print.ss, read.ss, record.ss, reloc.ss,
    strnum.ss, syntax.ss, trace.ss

original commit: b7f161bf2939dfedce8accbfa82b92dbe011d32a
2020-03-04 16:53:35 -05:00
Matthew Flatt
995e53ca71 Merge github.com:cisco/ChezScheme
original commit: 8cf52012e2a7b5928cb2602bb17e0128ae0f2776
2020-02-22 15:18:47 -07:00
dybvig
d0b405ac8b library-manager, numeric, and bytevector-compres improvements
- added invoke-library
    syntax.ss, primdata.ss,
    8.ms, root-experr*,
    libraries.stex, release_notes.stex
- updated the date
    release_notes.stex
- libraries contained within a whole program or library are now
  marked pending before their invoke code is run so that invoke
  cycles are reported as such rather than as attempts to invoke
  while still loading.
    compile.ss, syntax.ss, primdata.ss,
    7.ms, root-experr*
- the library manager now protects against unbound references
  from separately compiled libraries or programs to identifiers
  ostensibly but not actually exported by (invisible) libraries
  that exist only locally within a whole program.  this is done by
  marking the invisibility of the library in the library-info and
  propagating it to libdesc records; the latter is checked upon
  library import, visit, and invoke as well as by verify-loadability.
  the import and visit code of each invisible no longer complains
  about invisibility since it shouldn't be reachable.
    syntax.ss, compile.ss, expand-lang.ss,
    7.ms, 8.ms, root-experr*, patch*
- documented that compile-whole-xxx's linearization of the
  library initialization code based on static dependencies might
  not work for dynamic dependencies.
    system.stex
- optimized bignum right shifts so the code (1) doesn't look at
  shifted-off bigits if the bignum is positive, since it doesn't
  need to know in that case if any bits are set; (2) doesn't look
  at shifted-off bigits if the bignum is negative if it determines
  that at least one bit is set in the bits shifted off the low-order
  partially retained bigit; (3) quits looking, if it must look, for
  one bits as soon as it finds one; (4) looks from both ends under
  the assumption that set bits, if any, are most likely to be found
  toward the high or low end of the bignum rather than just in the
  middle; and (5) doesn't copy the retained bigits and then shift;
  rather shifts as it copies.  This leads to dramatic improvements
  when the shift count is large and often significant improvements
  otherwise.
    number.c,
    5_3.ms,
    release_notes.stex
- threaded tc argument through to all calls to S_bignum and
  S_trunc_rem so they don't have to call get_thread_context()
  when it might already have been called.
    alloc.c, number.c, fasl.c, print.c, prim5.c, externs.h
- added an expand-primitive handler to partially inline integer?.
    cpnanopass.ss
- added some special cases for basic arithmetic operations (+, -, *,
  /, quotient, remainder, and the div/div0/mod/mod0 operations) to
  avoid doing unnecessary work for large bignums when the result
  will be zero (e.g,. multiplying by 0), the same as one of the
  inputs (e.g., adding 0 or multiplying by 1), or the additive
  inverse of one of the inputs (e.g., subtracting from 0, dividing
  by -1).  This can have a major beneficial affect when operating
  on large bignums in the cases handled.  also converted some uses
  of / into integer/ where going through the former would just add
  overhead without the possibility of optimization.
    5_3.ss,
    number.c, externs.h, prim5.c,
    5_3.ms, root-experr, patch*,
    release_notes.stex
- added a queue to hold pending signals for which handlers have
  been registered via register-signal-handler so up to 63 (configurable
  in the source code) unhandled signals are buffered before the
  handler has to start dropping them.
    cmacros.ss, library.ss, prims.ss, primdata.ss,
    schsig.c, externs.h, prim5.c, thread.c, gc.c,
    unix.ms,
    system.stex, release_notes.stex
- bytevector-compress now selects the level of compression based
  on the compress-level parameter.  Prior to this it always used a
  default setting for compression.  the compress-level parameter
  can now take on the new minimum in addition to low, medium, high,
  and maximum.  minimum is presently treated the same as low
  except in the case of lz4 bytevector compression, where it
  results in the use of LZ4_compress_default rather than the
  slower but more effective LZ4_compress_HC.
    cmacros,ss, back.ss,
    compress_io.c, new_io.c, externs.h,
    bytevector.ms, mats/Mf-base, root-experr*
    io.stex, objects.stex, release_notes.stex

original commit: 72d90e4c67849908da900d0b6249a1dedb5f8c7f
2020-02-21 13:48:47 -08:00
Jamie Taylor
3e1ee3c681 Add concatenate-object-files procedure.
In previous versions of Chez Scheme, multiple object files could be
combined by concatinating them into a single file.  To support faster
object file loading and loadability verification, recompile information
and information about libraries and top-level programs within an object
file is now placed at the top of the file.  The new
concatenate-object-files procedure can be used to combine multiple object
files while moving this information to the top of the combined file.

original commit: d4ef2ad9393578ff3ffe3b712736bc6a4ae7b8eb
2020-02-18 14:51:38 -05:00
Matthew Flatt
d13e5898c8 add enable-unsafe-variable-reference
original commit: 77a0ef249b6d419dcac7c8e1c28e710b30a7aa33
2020-01-30 17:46:30 -07:00
Matthew Flatt
754bae07e3 add enable-unsafe-application
original commit: 4c0750d292999dbc476b2b0a80cad3b8beaab660
2020-01-23 15:25:22 -07:00
dybvig
48db0a9405 various library-manager improvements including the ability to verify
loadability without actually loading; also, support for unregistering
guarded objects.
- improved error reporting for library compilation-instance errors:
  now including the name of the object file from which the "wrong"
  compilation instance was loaded, if it was loaded from (or compiled
  to) an object file and the original importing library, if it was
  previously loaded from an object file due to a library import.
    syntax.ss, 7.ss, interpret.ss,
    8.ms, root-experr*
- removed situation and for-input? arguments from $make-load-binary,
  since the only consumer always passes 'load and #f.
    7.ss,
    scheme.c
- $separate-eval now prints the stderr and stdout of the subprocess
  to help in diagnosing separate-eval and separate-compile issues.
    mat.ss
- added unregister-guardian, which can be used to unregister
  the unressurected objects registered with any guardian.  guardian?
  can be used to distinguish guardian procedures from other objects.
    cp0.ss, cmacros.ss, cpnanopass.ss, ftype.ss, primdata.ss,
    prims.ss,
    gcwrapper.c, prim.c, externs.h,
    4.ms, primvars.ms
    release_notes.stex
    smgmt.stex, threads.stex
- added verify-loadability.  given a situation (visit, revisit,
  or load) and zero or more pathnames (each of which may be optionally
  paired with a library search path), verity-loadability checks
  whether the set of object files named by those pathnames and any
  additional object files required by library requirements in the
  given situation can be loaded together.  it raises an exception
  in each case where actually attempting to load the files would
  raise an exception and additionally in cases where loading files
  would result in the compilation or loading of source files in
  place of the object files.  if the check is successful,
  verity-loadability returns an unspecified value.  in either case,
  although portions of the object files are read, none of the
  information read from the object files is retained, and none of
  the object code is read, so there are no side effects other than
  the file operations and possibly the raising of an exception.
  library and program info records are now moved to the top of each
  object file produced by one of the file compilation routines,
  just after recompile info, with a marker to allow verity-loadability
  to stop reading once it reads all such records.  this change is
  not entirely backward compatible; the repositioning of the records
  can be detected by a call to list-library made from a loaded file
  before the definition of one or more libraries.  it is fully
  backward compatible for typical library files that contain a
  single library definition and nothing else.  adding this feature
  required changes to the object-file format and corresponding
  changes in the compiler and library manager.  it also required
  moving cross-library optimization information from library/ct-info
  records (which verity-loadability must read) to the invoke-code
  for each library (which verity-loadability  does not read) to
  avoid reading and permanently associating record-type descriptors
  in the code with their uids.
    compile.ss, syntax.ss, expand-lang.ss, primdata.ss, 7.ss,
    7.ms, misc.ms, root-experr*, patch*,
    system.stex, release_notes.stex
- fixed a bug that bit only with the compiler compiled at
  optimize-level 2: add-library/rt-records was building a library/ct-info
  wrapper rather than a library/rt-info wrapper.
    compile.ss
- fixed a bug in visit-library that could result in an indefinite
  recursion: it was not checking to make sure the call to $visit
  actually added compile-time info to the libdesc record.  it's not
  clear, however, whether the libdesc record can be missing
  compile-time information on entry to visit-library, so the code
  that calls $visit (and now checks for compile-time information
  having been added) might not be reachable.  ditto for
  revisit-library.
    syntax.ss
    syntax.ss, primdata.ss,
    7.ms, root-experr*, patch*,
    system.stex, release_notes.stex
- added some argument-error checks for library-directories and
  library-extensions, and fixed up the error messages a bit.
    syntax.ss,
    7.ms, root-experr*
- compile-whole-program now inserts the program record into the
  object file for the benefit of verify-loadability.
    syntax.ss,
    7.ms, root-experr*
- changed 'loading' import-notify messages to the more precise
  'visiting' or 'revisiting' in a couple of places.
    syntax.ss,
    7.ms, 8.ms

original commit: b911ed47190727b0e1d6a88c0e473d1757accdcd
2020-01-23 10:43:17 -08:00
Matthew Flatt
7e36c21d81 document enable-type-recovery
original commit: 30033287ad504058bc6810b1ef4ff5ecbe927dac
2019-11-26 07:15:16 -07:00
Matthew Flatt
232b230e65 add enable-error-source-expression
original commit: 02f383510653bb16c65fc812eb0c619bcb0347b5
2019-11-26 06:31:48 -07:00
Bob Burger
899a688bb0 fixed typos
original commit: e032c9ed09beccc3de827523aa1dc8b93ad5fc94
2019-09-23 16:02:23 -04:00
dybvig
7d145e37a8 Various enhancements and fixes highlighted by profiling performance
and functionality improvements (including support for measuring
coverage), primitive argument-checking fixes, and object-file changes
resulting in reduced load times (and some backward incompatibility):
- annotations are now preserved in object files for debug
  only, for profiling only, for both, or not at all, depending
  on the settings of generate-inspector-information and
  compile-profile.  in particular, when inspector information
  is not enabled but profiling is, source information does
  not leak into error messages and inspector output, though it is
  still available via the profile tools.  The mechanics of this
  involved repurposing the fasl a? parameter to hold an annotation
  flags value when it is not #f and remaking annotations with
  new flags if necessary before emitting them.
    compile.ss, fasl.ss, misc.ms
- altered a number of mats to produce correct results even
  when the 's' directory is profiled.
    misc.ms, cp0.ms, record.ms
- profile-release-counters is now generation-friendly; that is,
  it doesn't look for dropped code objects in generations that have
  not been collected since the last call to profile-release-counters.
  also, it no longer allocates memory when it releases counters.
    pdhtml.ss,
    gc.c, gcwrapper.c, globals.h, prim5.c
- removed unused entry points S_ifile, S_ofile, and S_iofile
    alloc.c, externs.h
- mats that test loading profile info into the compiler's database
  to guide optimization now weed out preexisting entries, in case
  the 's' directory is profiled.
    4.ms, mat.ss, misc.ms, primvars.ms
- counters for dropped code objects are now released at the start
  of each mat group.
    mat.ss
- replaced ehc (enable-heap-check) option with hci (heap-check-interval)
  option that allows heap checks to be performed periodically rather
  than on each collection.  hci=0 is equivalent to ehc=f (disabling
  heap checks) and hci=1 is equivalent to ehc=t (enabling heap
  checks every collection), while hci=100 enables heap checks only
  every 100th collection.  allx and bullyx mats use this feature
  to reduce heap-checking overhead to a more reasonable level.  this
  is particularly important when the 's' directory is profiled,
  since the amount of static memory to be checked is greatly increased
  due to the counters.
    mats/Mf-base, mat.ss, primvars.ms
- added a mat that calls #%show-allocation, which was otherwise not
  being tested.
    misc.ms
- removed a broken primvars mat and updated two others.  in each case,
  the mat was looking for information about primitives in the wrong
  (i.e., old) place and silently succeeding when it didn't find any
  primitives to tests.  the revised mats (along with a few others) now
  check to make sure at least one identifier has the information they
  look for.  the removed mat was checking for library information that
  is now compiled in, so the mat is now unnecessary.  the others were
  (not) doing argument-error checks.  fixing these turned up a handful of
  problems that have also been fixed: a couple of unbound variables in the
  mat driver, two broken primdata declarations, a tardy argument check
  by profile-load-data, and a bug in char-ready?, which was requiring
  an argument rather than defaulting it to the current input port.
    primdata.ss, pdhtml.ss, io.ms,
    primdvars.ms, 4.ms, 6.ms, misc.ms, patch*
- added initial support for recording coverage information.  when the
  new parameter generate-covin-files is set, the compiler generates
  .covin files containing the universe of all source objects for which
  profile forms are present in the expander output.  when profiling
  and generation of covin files are enabled in the 's' directory, the
  mats optionally generate .covout files for each mat file giving
  the subset of the universe covered by the mat file, along with an
  all.covout in each mat output directory aggregating the coverage
  for the directory and another all.covout in the top-level mat
  directory aggregating the coverage for all directories.
    back.ss, compile.ss, cprep.ss, primdata.ss, s/Mf-base,
    mat.ss, mats/Mf-base, mats/primvars.ms
- support for generating covout files is now built in.  with-coverage-output
  gathers and dumps coverage information, and aggregate-coverage-output
  combines (aggregates) covout files.
    pdhtml.ss, primdata.ss, compile.ss,
    mat.ss, mats/Mf-base, primvars.ms
- profile-clear now adjusts active coverage trackers to avoid losing
  coverage information.
    pdhtml.ss,
    prim5.c
- nested with-coverage calls are now supported.
    pdhtml.ss
- switched to a more compact representation for covin and covout files;
  reduces disk space (compressed or not) by about a factor of four
  and read time by about a factor of two with no increase in write time.
    primdata.ss, pdhtml.ss, cprep.ss, compile.ss,
    mat.ss, mats/Mf-base
- added support for determining coverage for an entire run, including
  coverage for expressions hit during boot time.  'all' mats now produce
  run.covout files in each output directory, and 'allx' mats produce
  an aggregate run.covout file in the mat directory.
    pdhtml.ss,
    mat.ss, mats/Mf-base
- profile-release-counters now adjusts active coverage trackers to
  account for the counters that have been released.
    pdhtml.ss,
    prim5.c
- replaced the artificial "examples" target with a real "build-examples"
  target so make won't think it always has to mats that depend upon
  the examples directory having been compiled.  mats make clean now
  runs make clean in the examples directory.
    mats/Mf-base
  importing a library from an object file now just visits the object
  file rather than doing a full load so that the run-time code for
  the library is not retained.  The run-time code is still read
  because the current fasl format forces the entire file to be read,
  but not retaining the code can lower heap size and garbage-collection
  cost, particularly when many object-code libraries are imported.
  The downside is that the file must be revisited if the run-time
  code turns out to be required.   This change exposed several
  places where the code was failing to check if a revisit is needed.
    syntax.ss,
    7.ms, 8.ms, misc.ms, root-experr*
- fixed typos: was passing unquoted load rather than quoted load
  to $load-library along one path (where it is loading source code
  and therefore irrelevant), and was reporting src-path rather than
  obj-path in a message about failing to define a library.
    syntax.ss
- compile-file and friends now put all recompile information in
  the first fasl object after the header so the library manager can
  find it without loading the entire fasl file.  The library manager
  now does so.  It also now checks to see if library object files
  need to be recreated before loading them rather than loading them and
  possibly recompiling them after discovering they are out of date, since
  the latter requires loading the full object file even if it's out of
  date, while the former takes advantage of the ability to extract just
  recompile information.  as well as reducing overhead, this eliminates
  possibly undesirable side effects, such as creation and registration
  of out-of-date nongenerative record-type descriptors.  because the
  library manager expects to find recompile information at the front of
  an object file, it will not find all recompile information if object
  files are "catted" together.  also, compile-file has to hold in memory
  the object code for all expressions in the file so that it can emit the
  unified recompile information, rather than writing to the object file
  incrementally, which can significantly increase the memory required
  to compile a large file full of individual top-level forms.  This does
  not affect top-level programs, which were already handled as a whole,
  or a typical library file that contains just a single library form.
    compile.ss, syntax.ss
- the library manager now checks include files before library dependencies
  when compile-imported-libraries is false (as it already did when
  compile-imported-libraries is true) in case a source change affects
  the set of imported libraries.  (A library change can affect the set
  of include files as well, but checking dependencies before include
  files can cause unneeded libraries to be loaded.)  The include-file
  check is based on recompile-info rather than dependencies, but the
  library checks are still based on dependencies.
    syntax.ss
- fixed check for binding of scheme-version. (the check prevents
  premature treatment of recompile-info records as Lexpand forms
  to be passed to $interpret-backend.)
    scheme.c
- strip-fasl-file now preserves recompile-info when compile-time info
  is stripped.
    strip.ss
- removed include-req* from library/ct-info and ctdesc records; it
  is no longer needed now that all recompile information is maintained
  separately.
    expand-lang.ss, syntax.ss, compile.ss, cprep.ss, syntax.ss
- changed the fasl format and reworked a lot of code in the expander,
  compiler, fasl writer, and fasl reader to allow the fasl reader
  to skip past run-time information when it isn't needed and
  compile-time information when it isn't needed.  Skipping past
  still involves reading and decoding when encrypted, but the fasl
  reader no longer parses or allocates code and data in the portions
  to be skipped.  Side effects of associating record uids with rtds
  are also avoided, as are the side effects of interning symbols
  present only in the skipped data.  Skipping past code objects
  also reduces or eliminates the need to synchronize data and
  instruction caches.  Since the fasl reader no longer returns
  compile-time (visit) or run-time (revisit) code and data when not
  needed, the fasl reader no longer wraps these objects in a pair
  with a 0 or 1 visit or revisit marker.  To support this change,
  the fasl writer generates separate top-level fasl entries (and
  graphs) for separate forms in the same top-level source form
  (e.g., begin or library).  This reliably breaks eq-ness of shared
  structure across these forms, which was previously broken only
  when visit or revisit code was loaded at different times (this
  is an incompatible change).  Because of the change, fasl "groups"
  are no longer needed, so they are no longer handled.
    7.ss, cmacros.ss, compile.ss, expand-lang.ss, strip.ss,
    externs.h, fasl.c, scheme.c,
    hash.ms
- the change above is surfaced in an optional fasl-read "situation"
  argument (visit, revisit, or load).  The default is load.  visit
  causes it to skip past revisit code and data; revisit causes it
  to skip past visit code and data; and load causes it not to skip
  past either.  visit-revisit data produced by (eval-when (visit
  revisit) ---) is never skipped.
    7.ss, primdata.ss,
    io.stex
- to improve compile-time and run-time error checking, the
  Lexpand recompile-info, library/rt-info, library-ct-info, and
  program-info forms have been replaced with list-structured forms,
  e.g., (recompile-info ,rcinfo).
    expand-lang.ss, compile.ss, cprep.ss, interpret.ss, syntax.ss
- added visit-compiled-from-port and revisit-compiled-from-port
  to complement the existing load-compiled-from-port.
    7.ss, primdata.ss,
    7.ms,
    system.stex
- increased amount read when seeking an lz4-encrypted input
  file from 32 to 1024 bytes at a time
    compress-io.c
- replaced the fasl a? parameter value #t with an "all" flag value
  so it's value is consistently a mask.
    cmacros.ss, fasl.ss, compile.ss
- split off profile mats into a separate file
    misc.ms, profile.ms (new), root-experr*, mats/Mf-base
- added coverage percent computations to mat allx/bullyx output
    mat.ss, mats/Mf-base, primvars.ms
- replaced coverage tables with more generic and generally useful
  source tables, which map source objects to arbitrary values.
    pdhtml.ss, compile.ss, cprep.ss, primdata.ss,
    mat.ss, mats/Mf-base, primvars.ms, profile.ms,
    syntax.stex
- reduced profile counting overhead by using calls to fold-left
  instead of calls to apply and map and by using fixnum operations
  for profile counts on 64-bit machines.
    pdhtml.ss
- used a critical section to fix a race condition in the calculations
  of profile counts that sometimes resulted in bogus (including
  negative) counts, especially when the 's' directory is profiled.
    pdhtml.ss
- added discard flag to declaration for hashtable-size
    primdata.ss
- redesigned the printed representation of source tables and rewrote
  get-source-table! to read and store incrementally to reduce memory
  overhead.
    compile.ss
- added generate-covin-files to the set of parameters preserved
  by compile-file, etc.
    compile.ss,
    system.stex
- moved covop argument before the undocumented machine and hostop
  arguments to compile-port and compile-to-port.  removed the
  undocumented ofn argument from compile-to-port; using
  (port-name ip) instead.
    compile.ss, primdata.ss,
    7.ms,
    system.stex
- compile-port now tries to come up with a file position to supply
  to make-read, which it can do if the port's positions are character
  positions (presently string ports) or if the port is positioned
  at zero.
    compile.ss
- audited the argument-type-error fuzz mat exceptions and fixed a
  host of problems this turned up (entries follow).  added #f as
  an invalid argument for every type for which #f is indeed invalid
  to catch places where the maybe- prefix was missing on the argument
  type.  the mat tries hard to determine if the condition raised
  (if any) as the result of an invalid argument is appropriate and
  redirects the remainder to the mat-output (.mo) file prefixed
  with 'Expected error', causing them to show up in the expected
  error output so developers will be encouraged to audit them in
  the future.
    primvars.ms, mat.ss
- added an initial symbol? test on machine type names so we produce
  an invalid machine type error message rather than something
  confusing like "machine type #f is not supported".
    compile.ss
- fixed declarations for many primitives that were specified as
  accepting arguments of more general types than they actually
  accept, such as number -> real for various numeric operations,
  symbol -> endianness for various bytevector operations,
  time -> time-utc for time-utc->date, and list -> list-of-string-pairs
  for default-library-search-handler.   also replaced some of the
  sub-xxxx types with specific types such as sub-symbol -> endianness
  in utf16->string, but only where they were causing issues with
  the primvars argument-type-error fuzz mat.  (this should be done
  more generally.)
    primdata.ss
- fixed incorrect who arguments (was map instead of fold-right,
  current-date instead of time-utc->date); switched to using
  define-who/set-who! generally.
    4.ss, date.ss
- append! now checks all arguments before any mutation
    5_2.ss
- with-source-path now properly supplies itself as who for the
  string? argument check; callers like load now do their own checks.
    7.ss
- added missing integer? check to $fold-bytevector-native-ref whose
  lack could have resulted in a compile-time error.
    cp0.ss
- fixed typo in output-port-buffer-mode error message
    io.ss
- fixed who argument (was fx< rather than fx<?)
    library.ss
- fixed declaration of first source-file-descriptor argument (was
  sfd, now string)
    primdata.ss
- added missing article 'a' in a few error messages
    prims.ss
- fixed the copy-environment argument-type error message for the list
  of symbols argument.
    syntax.ss
- the environment procedure now catches exceptions that occur and
  reraises the exception with itself as who if the condition isn't
  already a who condition.
    syntax.ss
- updated experr and allx patch files for changes to argument-count
  fuzz mat and fixes for problems turned up by them.
    root-experr*, patch*
- fixed a couple of issues setting port sizes: string and bytevector
  output port put handlers don't need room to store the character
  or byte, so they now set the size to the buffer length rather
  than one less.  binary-file-port-clear-output now sets the index
  rather than size to zero; setting the size to zero is inappropriate
  for some types of ports and could result in loss of buffering and
  even suppression of future output.  removed a couple of redundant
  sets of the size that occur immediately after setting the buffer.
    io.ss
- it is now possible to return from a call to with-profile-tracker
  multiple times and not double-count (or worse) any counts.
    pdhtml.ss, profile.ms
- read-token now requires a file position when it is handed a
  source-file descriptor (since the source-file descriptor isn't
  otherwise useful), and the source-file descriptor argument can
  no longer be #f.  the input file position plays the same role as
  the input file position in get-datum/annotations.  these extra
  read-token arguments are now documented.
    read.ss,
    6.ms,
    io.stex
- the source-file descriptor argument to get-datum/annotations can
  no longer be #f.  it was already documented that way.
    read.ss
- read-token and do-read now look for the character-positions port
  flag before asking if the port has port-position, since the latter
  is slightly more expensive.
    read.ss
- rd-error now reports the current port position if it can be determined
  when fp isn't already set, i.e., when reading from a port without
  character positions (presently any non string port) and fp has not
  been passed in explicitly (to read-token or get-datum/annotations).
  the port position might not be a character position, but it should be
  better than nothing.
    read.ss
- added comment noting an invariant for s_profile_release_counters.
    prim5.c
- restored accidentally dropped fasl-write formdef and dropped
  duplicate fasl-read formdef
    io.stex
- added a 'coverage' target that tests the coverage of the Scheme-code
  portions of Chez Scheme by the mats.
    Makefile.in, Makefile-workarea.in
- added .PHONY declarations for all of the targets in the top-level
  and workarea make files, and renamed the create-bintar, create-rpm,
  and create-pkg targets bintar, rpm, and pkg.
    Makefile.in, Makefile-workarea.in
- added missing --retain-static-relocation command-line argument and
  updated the date
    scheme.1.in
- removed a few redundant conditional variable settings
    configure
- fixed declaration of condition wait (timeout -> maybe-timeout)
    primdata.ss

original commit: 88501743001393fa82e89c90da9185fc0086fbcb
2019-09-21 15:37:29 -07:00
Matthew Flatt
ce9df2f827 Merge github.com:cisco/ChezScheme
original commit: c5d71168eb4315f7e8ec9c0acf615fa0b9a2fc88
2019-07-26 04:29:00 -06:00
Alexander Shopov
8c891262a1 Use setenv rather than putenv on non WIN32 environments
Signed-off-by: Alexander Shopov <ash@kambanaria.org>

original commit: 8bf1e18853d5feeb64aadb631c35641cd0ab4748
2019-07-25 16:06:48 +02:00
Matthew Flatt
2cf27c4727 Merge github.com:cisco/ChezScheme
original commit: 8118200e237d756f83be54e8bf3eabb4af2388ed
2019-05-22 10:46:59 -06:00
dyb
82b2cda639 compress-level parameter, improvement in lz4 compression, and various other related improvements
- added compress-level parameter to select a compression level for
  file writing and changed the default for lz4 compression to do a
  better job compressing.  finished splitting glz input routines
  apart from glz output routines and did a bit of other restructuring.
  removed gzxfile struct-as-bytevector wrapper and moved its fd
  into glzFile.  moved DEACTIVATE to before glzdopen_input calls
  in S_new_open_input_fd and S_compress_input_fd, since glzdopen_input
  reads from the file and could block.  the compress format and now
  level are now recorded directly the thread context.  replaced
  as-gz? flag bit in compressed bytevector header word with a small
  number of bits recording the compression format at the bottom of
  the header word.  flushed a couple of bytevector compression mats
  that depended on the old representation.  (these last few changes
  should make adding new compression formats easier.)  added
  s-directory build options to choose whether to compress and, if
  so, the format and level.
    compress-io.h, compress-io.c, new-io.c, equates.h, system.h,
    scheme.c, gc.c,
    io.ss, cmacros.ss, back.ss, bytevector.ss, primdata.ss, s/Mf-base,
    io.ms, mat.ss, bytevector.ms, root-experr*,
    release_notes.stex, io.stex, system.stex, objects.stex
- improved the effectiveness of LZ4 boot-file compression to within
  15% of gzip by increasing the lz4 output-port in_buffer size to
  1<<18.  With the previous size (1<<14) LZ4-compressed boot files
  were about 50% larger.  set the lz4 input-port in_buffer and
  out_buffer sizes to 1<<12 and 1<<14.  there's no clear win at
  present for larger input-port buffer sizes.
    compress-io.c
- To reduce the memory hit for the increased output-port in_buffer
  size and the corresponding increase in computed out_buffer size,
  one output-side out_buffer is now allocated (lazily) per thread
  and stored in the thread context.  The other buffers are now
  directly a part of the lz4File_out and lz4File_in structures
  rather than allocated separately.
    compress-io.c, scheme.c, gc.c,
    cmacros.ss
- split out the buffer emit code from glzwrite_lz4 into a
  separate glzemit_lz4 helper that is now also used by gzclose
  so we can avoid dealing with a NULL buffer in glzwrite_lz4.
  glzwrite_lz4 also uses it to writing large buffers directly and
  avoid the memcpy.
    compress-io.c
- replaced lz4File_out and lz4File_in mode enumeration with the
  compress format and inputp boolean.  using switch to check and
  raising exceptions for unexpected values to further simplify
  adding new compression formats in the future.
    compress-io.c
- replaced the never-defined struct lz4File pointer in glzFile
  union with the more specific struct lz4File_in_r and Lz4File_out_r
  pointers.
    compress-io.h, compress-io.c
- added free of lz4 structures to gzclose.  also changed file-close
  logic generally so that (1) port is marked closed before anything is
  freed to avoid dangling pointers in the case of an interrupt or
  error, and (2) structures are freed even in the case of a write
  or close error, before the error is reported.  also now mallocing
  glz and lz4 structures after possibility of errors have passed where
  possible and freeing them when not.
    compress-io.c,
    io.ss
- added return-value checks to malloc calls and to a couple of other
  C-library calls.
    compress-io.c
- corrected EINTR checks to look at errno rather than return codes.
    compress-io.c
- added S_ prefixes to the glz* exports
    externs.h, compress-io.c, new-io.c, scheme.c, fasl.c
- added entries for mutex-name and mutex-thread
    threads.stex

original commit: 722ffabef4c938bc92c0fe07f789a9ba350dc6c6
2019-04-18 05:47:19 -07:00
Matthew Flatt
8070a7b910 Merge branch 'eqfl' of github.com:mflatt/ChezScheme
original commit: 8b36396eacb139e0fff70efcd2c9dc842815324f
2019-01-22 05:57:17 -07:00
Matthew Flatt
67a0b0948f Merge branch 'leftassoc' of github.com:mflatt/ChezScheme
original commit: 19d0cb77e6b6dd32fa4a7a26505ee952b4efa756
2019-01-21 10:54:19 -07:00
Matthew Flatt
ae71f7472d add enable-arithmetic-left-associative
Add a parameter to constrain the compiler (off by default) to
implement `+`, `*`, and variants as left-associative when given
multiple arguments.

original commit: d126ba3364893e66263c65af1cd6dbdd8b021439
2019-01-21 10:51:16 -07:00
Matthew Flatt
b27f3c0a94 Merge branch 'phantom' of github.com:mflatt/ChezScheme
original commit: 743a56d8f1920620e8f6e14edca7984101425e14
2019-01-20 07:56:59 -07:00
Matthew Flatt
538def47de add phantom bytevectors
original commit: 001917fd98ac6a0f13ccab902e15b9d2169c4b9c
2019-01-20 07:41:09 -07:00
dyb
ee9a4b3f59 profile counts are now maintained even for code that has been
reclaimed by the collector and must be released explicitly by the
programmer via (profile-release-counters).
  pdhtml.ss, primdata.ss,
  globals.h, externs.h, fasl.c, prim5.c, prim.c, alloc.c, scheme.c,
  misc.ms,
  release_notes.stex, system.stex

original commit: 68e20f721618dbaf4c1634067c2bee24a493a750
2019-01-17 09:43:18 -08:00
Matthew Flatt
7c548bb3a1 update vfasl merge
original commit: 99dac3f53f4a7d2b2c373489135e5d270c256726
2018-12-28 08:39:21 -06:00
Matthew Flatt
ed1d5c982d Merge ../ChezScheme-vfasl
original commit: 78ba118cbde76dd42bc4275ccc76219e159e04d7
2018-12-20 17:51:38 -07:00
Matthew Flatt
efb93d2653 Merge branch 'gcbt' of github.com:mflatt/ChezScheme
original commit: 51c6b2a880000ce754e1595f4481957e9fc7f722
2018-12-16 07:00:22 -07:00
Oscar Waddell
36ea9a354d fix typos
original commit: 64a4374513fa19123147c6ef5644f75177a77541
2018-08-30 07:51:27 -04:00
Matthew Flatt
755a9252c5 Merge branch 'genid' of github.com:mflatt/ChezScheme
original commit: 0ded12c942cb47f3d9cb7d255fab1b07874f139c
2018-07-25 16:05:48 -06:00
Matthew Flatt
1860fec019 Merge branch 'gcbt' of github.com:mflatt/ChezScheme
original commit: 59bd17ca5713fbe913e9297e7cbb719b2c45ca7d
2018-07-25 16:01:11 -06:00
Bob Burger
be2777e261 fix boot_call and the invoke code object to handle multiple values
original commit: 7fd080903d96a1ef9ab79780bf73aa1619f33894
2018-07-18 14:55:53 -04:00
Bob Burger
02131784f9 Merge branch 'procloc' of https://github.com/mflatt/ChezScheme into procloc
# Conflicts:
#	LOG
#	release_notes/release_notes.stex

original commit: 67b326cb9199a021d7c025834dedc8ac6285fc46
2018-07-18 10:18:24 -04:00
Matthew Flatt
48228739fe add object-references to reflect GC's tracing of objects
The `object-references` function is intended to support debugging of
memory leaks by providing a mapping from each live object to the
object that retained it.

original commit: 61f6602b7e6c388c529f3c5995dcf71a7c42e005
2018-07-16 18:08:48 -06:00
Matthew Flatt
067555c251 add load-compiled-from-port and Sregister_boot_file_fd
original commit: a0adfa18af879f90d746b0b3541b036016957324
2018-07-15 20:44:34 -06:00
Matthew Flatt
6dd593ed89 add current-generate-id
original commit: 1e4cff5631e9e6615528b60ebcf3c04a8f38e3a0
2018-07-15 20:22:15 -06:00
Matthew Flatt
9521af1518 add generate-procedure-source-information
original commit: b42213a65e75b29f51d29a77d258fa1ba8a336b0
2018-07-15 19:51:54 -06:00
Bob Burger
ad09be1a6a Updated some copyright years in csug
original commit: 6c738bdfde601b815c296f489697e61c314e4621
2018-06-21 13:44:05 -04:00
Bob Burger
8885445d6d Improved Unicode support for command-line arguments, environment variables, the C interface and error messages, and the Windows registry, DLL loading, and process creation
original commit: aa1c2c4ec95c286a12730ea75588a18dd9fb9d59
2018-06-14 14:24:15 -04:00
Andy Keep
f673aeb57e Small fixes to documentation.
Fixed typo in profile-dump-html index and corrected overfull box issue in pdf
version of docs.
  foreign.stex, system.stex

original commit: 3a3385c7d62326ba3d0d3220329dfcd3e3d0ac8e
2018-05-22 21:57:49 -04:00
Oscar Waddell
3a5a3ab751 fix duplicate words
original commit: 38ade9595fe9771a02b00e22a50cede215a6b9ce
2018-05-22 16:09:14 -04:00
Oscar Waddell
0fa4928592 fix typos
original commit: 014d199a83960f58df1fb2da4c4849b162fc59e1
2018-05-22 16:09:00 -04:00
dybvig
f7c414bda3 Various updates, mostly to the compiler, including a new lambda
commonizatio pass and support for specifying default record
equal and hash procedures:

- more staid and consistent Mf-cross main target
    Mf-cross
- cpletrec now replaces the incoming prelexes with new ones so
  that it doesn't have to alter the flags on the incoming ones, since
  the same expander output is passed through the compiler twice while
  compiling a file with macro definitions or libraries.  we were
  getting away without this just by luck.
    cpletrec.ss
- pure? and ivory? now return #t for a primref only if the prim is
  declared to be a proc, since some non-proc prims are mutable, e.g.,
  $active-threads and $collect-request-pending.
    cp0.ss
- $error-handling-mode? and $eol-style? are now properly declared to
  be procs rather than system state variables.
    primdata.ss
- the new pass $check-prelex-flags verifies that prelex referenced,
  multiply-referenced, and assigned flags are set when they
  should be.  (it doesn't, however, complain if a flag is set
  when it need not be.)  when the new system parameter
  $enable-check-prelex-flags is set, $check-prelex-flags is
  called after each major pass that produces Lsrc forms to verify
  that the flags are set correctly in the output of the pass.
  this parameter is unset by default but set when running the
  mats.
    cprep.ss, back.ss, compile.ss, primdata.ss,
    mats/Mf-base
- removed the unnecessary set of prelex referenced flag from the
  build-ref routines when we've just established that it is set.
    syntax.ss, compile.ss
- equivalent-expansion? now prints differences to the current output
  port to aid in debugging.
    mat.ss
- the nanopass that patches calls to library globals into calls to
  their local counterparts during whole-program optimization now
  creates new prelexes and sets the prelex referenced, multiply
  referenced, and assigned flags on the new prelexes rather than
  destructively setting flags on the incoming prelexes.  The
  only known problems this fixes are (1) the multiply referenced
  flag was not previously being set for cross-library calls when
  it should have been, resulting in overly aggressive inlining
  of library exports during whole-program optimization, and (2)
  the referenced flag could sometimes be set for library exports
  that aren't actually used in the final program, which could
  prevent some unreachable code from being eliminated.
    compile.ss
- added support for specifying default record-equal and
  record-hash procedures.
    primdata.ss, cmacros.ss, cpnanopass.ss, prims.ss, newhash.ss,
    gc.c,
    record.ms
- added missing call to relocate for subset-mode tc field, which
  wasn't burning us because the only valid non-false value, the
  symbol system, is in the static generation after the initial heap
  compaction.
    gc.c
- added a lambda-commonization pass that runs after the other
  source optimizations, particularly inlining, and a new parameter
  that controls how hard it works.  the value of commonization-level
  ranges from 0 through 9, with 0 disabling commonization and 9
  maximizing it.  The default value is 0 (disabled).  At present,
  for non-zero level n, the commonizer attempts to commonize
  lambda expressions consisting of 2^(10-n) or more nodes.
  commonization of one or more lambda expressions requires that
  they have identical structure down to the leaf nodes for quote
  expressions, references to unassigned variables, and primitives.
  So that various downstream optimizations aren't disabled, there
  are some additional restrictions, the most important of which
  being that call-position expressions must be identical.  The
  commonizer works by abstracting the code into a helper that
  takes the values of the differing leaf nodes as arguments.
  the name of the helper is formed by concatenating the names of
  the original procedures, separated by '&', and this is the name
  that will show up in a stack trace.  The source location will
  be that of one of the original procedures.  Profiling inhibits
  commonization, because commonization requires profile source
  locations to be identical.
    cpcommonize.ss (new), compile.ss, interpret.ss, cprep.ss,
    primdata.ss, s/Mf-base,
    mats/Mf-base
- cpletrec now always produces a letrec rather than a let for
  single immutable lambda bindings, even when not recursive, for
  consistent expand/optimize output whether the commonizer is
  run or not.
    cpletrec.ss,
    record.ms
- trans-make-ftype-pointer no longer generates a call to
  $verify-ftype-address if the address expression is a call to
  ftype-pointer-address.
    ftype.ss

original commit: b6a3dcc814b64faacc9310fec4a4531fb3f18dcd
2018-01-29 09:20:07 -05:00
dybvig
f559e220db - minor wordsmithing and fix for an overfull hbox
objects.stex, system.stex

original commit: dfa469217af6ddea0bc68086b55a22b253977dbf
2017-07-12 13:40:49 -04:00
Sam Tobin-Hochstadt
0e41c9d8be Add date-dst? and date-zone-name procedures
Accesses existing-but-hidden DST information in date records,
and adds support for getting a name of the current time zone
when a time zone offset is not explicitly provided.

original commit: 3c8be62d12a8197018fb6c5aae182fd75da14fe8
2017-06-08 11:21:39 -04:00
Bob Burger
a618b8860b fixed inverted logic in description of maybe-compile-*
original commit: 3747716ba6a11073146619ce75ff625ad8cbd7c0
2017-04-13 12:59:39 -04:00
Bob Burger
ee0368b462 accross => across in csug/system.stex
original commit: 37b84d9c6085d818ff993b4acc97746762997330
2017-04-13 12:48:36 -04:00
Bob Burger
831ea8ad18 changed copyright year to 2017
7.ss, scheme.1.in, comments of many files

original commit: 06f858f9a505b9d6fb6ca1ac97234927cb2dc641
2017-04-06 11:41:33 -04:00
dybvig
fde2de6249 Merge branch 'master' of git://github.com/michaellenaghan/ChezScheme into michaellenaghan-master
original commit: cdb20639749b7a38f85629a1db46b29624a588ad
2016-07-30 14:36:14 -04:00