Commit Graph

9 Commits

Author SHA1 Message Date
dybvig
d0b405ac8b library-manager, numeric, and bytevector-compres improvements
- added invoke-library
    syntax.ss, primdata.ss,
    8.ms, root-experr*,
    libraries.stex, release_notes.stex
- updated the date
    release_notes.stex
- libraries contained within a whole program or library are now
  marked pending before their invoke code is run so that invoke
  cycles are reported as such rather than as attempts to invoke
  while still loading.
    compile.ss, syntax.ss, primdata.ss,
    7.ms, root-experr*
- the library manager now protects against unbound references
  from separately compiled libraries or programs to identifiers
  ostensibly but not actually exported by (invisible) libraries
  that exist only locally within a whole program.  this is done by
  marking the invisibility of the library in the library-info and
  propagating it to libdesc records; the latter is checked upon
  library import, visit, and invoke as well as by verify-loadability.
  the import and visit code of each invisible no longer complains
  about invisibility since it shouldn't be reachable.
    syntax.ss, compile.ss, expand-lang.ss,
    7.ms, 8.ms, root-experr*, patch*
- documented that compile-whole-xxx's linearization of the
  library initialization code based on static dependencies might
  not work for dynamic dependencies.
    system.stex
- optimized bignum right shifts so the code (1) doesn't look at
  shifted-off bigits if the bignum is positive, since it doesn't
  need to know in that case if any bits are set; (2) doesn't look
  at shifted-off bigits if the bignum is negative if it determines
  that at least one bit is set in the bits shifted off the low-order
  partially retained bigit; (3) quits looking, if it must look, for
  one bits as soon as it finds one; (4) looks from both ends under
  the assumption that set bits, if any, are most likely to be found
  toward the high or low end of the bignum rather than just in the
  middle; and (5) doesn't copy the retained bigits and then shift;
  rather shifts as it copies.  This leads to dramatic improvements
  when the shift count is large and often significant improvements
  otherwise.
    number.c,
    5_3.ms,
    release_notes.stex
- threaded tc argument through to all calls to S_bignum and
  S_trunc_rem so they don't have to call get_thread_context()
  when it might already have been called.
    alloc.c, number.c, fasl.c, print.c, prim5.c, externs.h
- added an expand-primitive handler to partially inline integer?.
    cpnanopass.ss
- added some special cases for basic arithmetic operations (+, -, *,
  /, quotient, remainder, and the div/div0/mod/mod0 operations) to
  avoid doing unnecessary work for large bignums when the result
  will be zero (e.g,. multiplying by 0), the same as one of the
  inputs (e.g., adding 0 or multiplying by 1), or the additive
  inverse of one of the inputs (e.g., subtracting from 0, dividing
  by -1).  This can have a major beneficial affect when operating
  on large bignums in the cases handled.  also converted some uses
  of / into integer/ where going through the former would just add
  overhead without the possibility of optimization.
    5_3.ss,
    number.c, externs.h, prim5.c,
    5_3.ms, root-experr, patch*,
    release_notes.stex
- added a queue to hold pending signals for which handlers have
  been registered via register-signal-handler so up to 63 (configurable
  in the source code) unhandled signals are buffered before the
  handler has to start dropping them.
    cmacros.ss, library.ss, prims.ss, primdata.ss,
    schsig.c, externs.h, prim5.c, thread.c, gc.c,
    unix.ms,
    system.stex, release_notes.stex
- bytevector-compress now selects the level of compression based
  on the compress-level parameter.  Prior to this it always used a
  default setting for compression.  the compress-level parameter
  can now take on the new minimum in addition to low, medium, high,
  and maximum.  minimum is presently treated the same as low
  except in the case of lz4 bytevector compression, where it
  results in the use of LZ4_compress_default rather than the
  slower but more effective LZ4_compress_HC.
    cmacros,ss, back.ss,
    compress_io.c, new_io.c, externs.h,
    bytevector.ms, mats/Mf-base, root-experr*
    io.stex, objects.stex, release_notes.stex

original commit: 72d90e4c67849908da900d0b6249a1dedb5f8c7f
2020-02-21 13:48:47 -08:00
dyb
82b2cda639 compress-level parameter, improvement in lz4 compression, and various other related improvements
- added compress-level parameter to select a compression level for
  file writing and changed the default for lz4 compression to do a
  better job compressing.  finished splitting glz input routines
  apart from glz output routines and did a bit of other restructuring.
  removed gzxfile struct-as-bytevector wrapper and moved its fd
  into glzFile.  moved DEACTIVATE to before glzdopen_input calls
  in S_new_open_input_fd and S_compress_input_fd, since glzdopen_input
  reads from the file and could block.  the compress format and now
  level are now recorded directly the thread context.  replaced
  as-gz? flag bit in compressed bytevector header word with a small
  number of bits recording the compression format at the bottom of
  the header word.  flushed a couple of bytevector compression mats
  that depended on the old representation.  (these last few changes
  should make adding new compression formats easier.)  added
  s-directory build options to choose whether to compress and, if
  so, the format and level.
    compress-io.h, compress-io.c, new-io.c, equates.h, system.h,
    scheme.c, gc.c,
    io.ss, cmacros.ss, back.ss, bytevector.ss, primdata.ss, s/Mf-base,
    io.ms, mat.ss, bytevector.ms, root-experr*,
    release_notes.stex, io.stex, system.stex, objects.stex
- improved the effectiveness of LZ4 boot-file compression to within
  15% of gzip by increasing the lz4 output-port in_buffer size to
  1<<18.  With the previous size (1<<14) LZ4-compressed boot files
  were about 50% larger.  set the lz4 input-port in_buffer and
  out_buffer sizes to 1<<12 and 1<<14.  there's no clear win at
  present for larger input-port buffer sizes.
    compress-io.c
- To reduce the memory hit for the increased output-port in_buffer
  size and the corresponding increase in computed out_buffer size,
  one output-side out_buffer is now allocated (lazily) per thread
  and stored in the thread context.  The other buffers are now
  directly a part of the lz4File_out and lz4File_in structures
  rather than allocated separately.
    compress-io.c, scheme.c, gc.c,
    cmacros.ss
- split out the buffer emit code from glzwrite_lz4 into a
  separate glzemit_lz4 helper that is now also used by gzclose
  so we can avoid dealing with a NULL buffer in glzwrite_lz4.
  glzwrite_lz4 also uses it to writing large buffers directly and
  avoid the memcpy.
    compress-io.c
- replaced lz4File_out and lz4File_in mode enumeration with the
  compress format and inputp boolean.  using switch to check and
  raising exceptions for unexpected values to further simplify
  adding new compression formats in the future.
    compress-io.c
- replaced the never-defined struct lz4File pointer in glzFile
  union with the more specific struct lz4File_in_r and Lz4File_out_r
  pointers.
    compress-io.h, compress-io.c
- added free of lz4 structures to gzclose.  also changed file-close
  logic generally so that (1) port is marked closed before anything is
  freed to avoid dangling pointers in the case of an interrupt or
  error, and (2) structures are freed even in the case of a write
  or close error, before the error is reported.  also now mallocing
  glz and lz4 structures after possibility of errors have passed where
  possible and freeing them when not.
    compress-io.c,
    io.ss
- added return-value checks to malloc calls and to a couple of other
  C-library calls.
    compress-io.c
- corrected EINTR checks to look at errno rather than return codes.
    compress-io.c
- added S_ prefixes to the glz* exports
    externs.h, compress-io.c, new-io.c, scheme.c, fasl.c
- added entries for mutex-name and mutex-thread
    threads.stex

original commit: 722ffabef4c938bc92c0fe07f789a9ba350dc6c6
2019-04-18 05:47:19 -07:00
Matthew Flatt
e622a495b6 Add LZ4 support and use it by default for compressing files
original commit: 8858b34bd92ac8d2b6511dc9ca17ebfa06a1bd93
2019-04-06 07:32:37 +02:00
dyb
2daf225cab committing a handful of changes, none of which should be particularly
controversial, unless I damaged something in the process of integrating
them with other recent changes.  the user's guide and release notes
have been updated as well to reflect the changes of interest to end
users.
- the body of load-library is now wrapped in a $pass-time with
  to show the time spent loading libraries separately from the time
  spent in expand.
    syntax.ss
- interpret now plays the pass-time game
    interpret.ss
- added compile-time-value? predicate and
  compile-time-value-value accessor
    syntax.ss, primdata.ss,
    8.ms, primvars.ms, root-experr*
- $pass-stats now returns accurrate stats for the currently timed
  pass.
    7.ss
- compile-whole-program and compile-whole-library now propagate
  recompile info from the named wpo file to the object file
  to support maybe-compile-program and maybe-compile-library in
  the case where compile-whole-{program,library} overwrites the
  original object file.
    compile.ss,
    7.ms, mat.ss, primvars.ms
- replaced the ancient and unusable bintar with one that creates
  a useful tarball for binary installs
    bintar
- generated Mf-install InstallBin (InstallLib, InstallMan) now
  correctly indirects through InstallPrefix if the --installbin
  (--installlib, --installman) configure flag is not present.
    src/configure
- removed definition of generate-procedure-source-information
    patch.ss
- guardian tconc cells are now allocated in generation 0 in the hope
  that they can be released more quickly.
    gc.c
- added ftype-guardian syntax: (ftype-guardian A) creates a new
  guardian for ftype pointers of type A, the first base field (or
  one of the first base fields in the case of unions) of which must
  be a word-sized integer with native endianness representing a
  reference count.  ftype pointers are registered with and retrieved
  from the guardian just like objects are registered with and
  retrieved from any guardian.  the difference is that the garbage
  collector decrements the reference count before resurrecting an
  ftype pointer and resurrects only those whose reference counts
  become zero, i.e., are ready for deallocation.
    ftype.ss, cp0.ss, cmacros.ss, cpnanopass.ss, prims.ss, primdata.ss,
    gc.c,
    4.ms, root-experr*
- fixed a bug in automatic recompilation handling of missing include
  files specified with absolute pathnames or pathnames starting with
  "./" or "..": was erroring out in file-modification-time with a
  file-not-found or other exception rather than recompiling.
    syntax.ss,
    7.ms, root-experr*, patch*
- changed inline vector-for-each and string-for-each code to
  put the last call to the procedure in tail position, as was
  already done for the library definitions and for the inline
  code for for-each.
    cp0.ss,
    5_4.ms, 5_6.ms
- the compiler now generates better inline code for the bytevector
  procedure.  instead of one byte memory write for each argument,
  it writes up to 4 (32-bit machines) or 8 (64-bit machines) bytes
  at a time, which almost always results in fewer instructions and
  fewer writes.
    cpnanopass.ss,
    bytevector.ms
- packaged unchanging implicit reader arguments into a single record
  to reduce the number of arguments.
    read.ss
- recoded run-vector to handle zero-length vectors.  it appears
  we're not presently generating empty vectors (representing empty
  groups), but the fasl format permits them.
    7.ss

original commit: 7be1d190de7171f74a1ee71e348d3e6310392686
2019-02-11 20:06:42 -08:00
Matthew Flatt
1dba8c4624 fix bounds check on immutables values and an immediate index
original commit: fc064b5b91bc5c7da8b7c71441a37e3c05523986
2018-01-07 08:48:57 -07:00
Matthew Flatt
1932612543 add bytevector-compress and bytevector-decompress
original commit: aa062c09c9f0d129250db84aeb0a5190647c5574
2017-06-21 17:52:28 -06:00
Bob Burger
831ea8ad18 changed copyright year to 2017
7.ss, scheme.1.in, comments of many files

original commit: 06f858f9a505b9d6fb6ca1ac97234927cb2dc641
2017-04-06 11:41:33 -04:00
Kent Dybvig
9cd0199a39 merge @mflatt immutable-vector, immutable-string, immutable-bytevector,
immutable-fxvector, and immutable-box support

original commit: 547fce9b99c6566d5cb3f7af9ca84654e798486e
2017-03-15 11:09:57 -04:00
dyb
1356af91b3 initial upload of open-source release
original commit: 47a210c15c63ba9677852269447bd2f2598b51fe
2016-04-26 10:04:54 -04:00