On x86_64, a POPCNT instruction is usually available, and it can speed
up `fxpopcount` operations by a factor of 2-3.
Since POPCNT isn't always available, code using `fxpopcount` is
compiled to a call to a generic implementation. The linker substitutes
a POPCNT instruction when it determines at runtime that POPCNT is
available.
Some measurements on a 2018 MacBook Pro (2.7 GHz Core i7) using the
program below:
popcnt = this implementation, POPCNT discovered
nocnt = this implementation, POPCNT considered unavailable
optcnt = compile to use POPCNT directly (no linker work)
cpcnt = compile to inlined generic (no linker work, no POPCNT)
Since the generic implementation is always a 64-bit popcount, it's not
as good as an inlined version for `fxpopcount32`, but otherwise the
link-edit approach to POPCNT works well:
fxpopcount fxpopcount32
popcnt: 0.098s
nocnt: 0.284s
optcnt 0.109s [slower means noise?]
cpcnt: 0.279s 0.188s
(optimize-level 3)
(time
(let loop ([v #f] [i 100000000])
(if (fx= i 0)
v
(loop (fxpopcount i) (fx- i 1)))))
original commit: 5f090e509f8fe5edc777ed9f0463b20c2e571336
Uninterned symbols are slightly more expensive to allocate than 0- or
1-argument calls to `gensym`, but they're much cheaper to hash (and
print). They're also more consistently distinct when unfasled, and the
fasled form is determinsitic.
original commit: 3167083008031b1f880e76a6f573563c7d9c888c
The MRG32k3a generator is fast when using unboxed floating-point
arithemtic. Since the Scheme compiler doesn't yet support that,
build MRG32k3a into the kernel and provide access via
`pseudo-random-generator` functions.
original commit: 3dd74679a6c2705440488d8c07c47852eb50a94b
If normal 1-shot continuations are mixed with opportunistic 1-shot
continuations created by `call-setting-continuation-attachment`, then
promoting an opportunistic 1-shot at a GC is wrong unless the whole
chain is promoted.
original commit: 2dfac475666763b60935e382386af4438f3029e0
Move `NORETURN` of 2e3a618b00 to start of function declaration, where
it works for both GCC and MSVC.
original commit: 10fc4a2406ecd34fa686d9d643ee63d7c12d6f97
Merging ".o" files to one "kernel.o" can be convenient for further
linking, but it requires running `ld` directly. Running `ld` directly
sometimes runs into a mismatch between the C compiler and the default
`ld`. It's better to use the more typical approach of collecting
objects into an archive.
original commit: 7d5b60c7566570655e567495d86d546101cf8fb4
- added compress-level parameter to select a compression level for
file writing and changed the default for lz4 compression to do a
better job compressing. finished splitting glz input routines
apart from glz output routines and did a bit of other restructuring.
removed gzxfile struct-as-bytevector wrapper and moved its fd
into glzFile. moved DEACTIVATE to before glzdopen_input calls
in S_new_open_input_fd and S_compress_input_fd, since glzdopen_input
reads from the file and could block. the compress format and now
level are now recorded directly the thread context. replaced
as-gz? flag bit in compressed bytevector header word with a small
number of bits recording the compression format at the bottom of
the header word. flushed a couple of bytevector compression mats
that depended on the old representation. (these last few changes
should make adding new compression formats easier.) added
s-directory build options to choose whether to compress and, if
so, the format and level.
compress-io.h, compress-io.c, new-io.c, equates.h, system.h,
scheme.c, gc.c,
io.ss, cmacros.ss, back.ss, bytevector.ss, primdata.ss, s/Mf-base,
io.ms, mat.ss, bytevector.ms, root-experr*,
release_notes.stex, io.stex, system.stex, objects.stex
- improved the effectiveness of LZ4 boot-file compression to within
15% of gzip by increasing the lz4 output-port in_buffer size to
1<<18. With the previous size (1<<14) LZ4-compressed boot files
were about 50% larger. set the lz4 input-port in_buffer and
out_buffer sizes to 1<<12 and 1<<14. there's no clear win at
present for larger input-port buffer sizes.
compress-io.c
- To reduce the memory hit for the increased output-port in_buffer
size and the corresponding increase in computed out_buffer size,
one output-side out_buffer is now allocated (lazily) per thread
and stored in the thread context. The other buffers are now
directly a part of the lz4File_out and lz4File_in structures
rather than allocated separately.
compress-io.c, scheme.c, gc.c,
cmacros.ss
- split out the buffer emit code from glzwrite_lz4 into a
separate glzemit_lz4 helper that is now also used by gzclose
so we can avoid dealing with a NULL buffer in glzwrite_lz4.
glzwrite_lz4 also uses it to writing large buffers directly and
avoid the memcpy.
compress-io.c
- replaced lz4File_out and lz4File_in mode enumeration with the
compress format and inputp boolean. using switch to check and
raising exceptions for unexpected values to further simplify
adding new compression formats in the future.
compress-io.c
- replaced the never-defined struct lz4File pointer in glzFile
union with the more specific struct lz4File_in_r and Lz4File_out_r
pointers.
compress-io.h, compress-io.c
- added free of lz4 structures to gzclose. also changed file-close
logic generally so that (1) port is marked closed before anything is
freed to avoid dangling pointers in the case of an interrupt or
error, and (2) structures are freed even in the case of a write
or close error, before the error is reported. also now mallocing
glz and lz4 structures after possibility of errors have passed where
possible and freeing them when not.
compress-io.c,
io.ss
- added return-value checks to malloc calls and to a couple of other
C-library calls.
compress-io.c
- corrected EINTR checks to look at errno rather than return codes.
compress-io.c
- added S_ prefixes to the glz* exports
externs.h, compress-io.c, new-io.c, scheme.c, fasl.c
- added entries for mutex-name and mutex-thread
threads.stex
original commit: 722ffabef4c938bc92c0fe07f789a9ba350dc6c6
reclaimed by the collector and must be released explicitly by the
programmer via (profile-release-counters).
pdhtml.ss, primdata.ss,
globals.h, externs.h, fasl.c, prim5.c, prim.c, alloc.c, scheme.c,
misc.ms,
release_notes.stex, system.stex
original commit: 68e20f721618dbaf4c1634067c2bee24a493a750
- when thread_get_room exhausts the local allocation area, it now
goes through a common path with S_get_more_room to allocate a new
local allocation area when appropriate. this can greatly reduce
the use of global allocation (and the number of tc mutex acquires
in threaded builds) when a lot of small objects are allocated by
C code with no intervening Scheme-side allocation or dirty writes.
alloc.c, types.h, externs.h
original commit: 93dfa7674a95837e5a22bc622fecc50b0224f60d
The `object-references` function is intended to support debugging of
memory leaks by providing a mapping from each live object to the
object that retained it.
original commit: 61f6602b7e6c388c529f3c5995dcf71a7c42e005
convention that the only undocumented externs are prefixed with
S_.
externs.h, stats.c, thread.c
original commit: 898c3941d807a18c393a4575a72283e3671ee7e5
allocating the thread context fails from Sactivate_thread. before
this change, the error was raised on the main thread, which resulted
in strange behavior at best. also added who argument to
S_create_thread_object to allow it to report either Sactivate_thread
or fork-thread led to the error.
externs.h, schsig.c, scheme.c, thread.c
original commit: 89691cee27ee7d5e9bffae530b6346e96f8cc7ad
sizes of pathnames produced by expansion of tilde (home-directory)
prefixes by replacing S_pathname, S_pathname_impl, and S_homedir
with S_malloc_pathname, which always mallocs space for the result.
one thread-safety issue involved the use of static strings for expanded
pathnames and affected various file-system operations. the other
affected the file open routines and involved use of the incoming
pathname while deactivated. the incoming pathname is sometimes if not
always a pointer into a Scheme bytevector, which can be overwritten if a
collection occurs while the thread is deactivated. the size limitation
corresponded to the use of the static strings, which were limited to
PATH_MAX bytes. (PATH_MAX typically isn't actually the maximum path
length in contemporary operating systems.) eliminated similar issues
for wide pathnames under Windows by adding S_malloc_wide_pathname.
consumers of the old routines have been modified to use the new
routines and to free the result strings. the various file operations
now consistently treat a pathname with an unresolvable home directory
as a pathname that happens to start with a tilde. eliminated unused
foreign-symbol binding of "(cs)pathname" to S_pathname.
io.c, externs.h, new_io.c, prim5.c, scheme.c, prim.c
- various places where a call to close or gzclose was retried when
the close operation was interrupted no longer do so, since this can
cause problems when another thread has reallocated the same file
descriptor.
new_io.c
- now using vcvarsall type x86_amd64 rather than amd64 when the
former appears to supported and the latter does not, as is the
case with VS Express 2015.
c/Mf-a6nt, c/Mf-ta6nt
- commented out one of the thread mats that consistently causes
indefinite delays under Windows and OpenBSD due to starvation.
thread.ms
- increased wait time for a couple of subprocess responses
6.ms
- added call to collector to close files opened during iconv mats
specifically for when mats are run under Windows with no iconv dll.
io.ms
original commit: ad44924307c576eb2fc92e7958afe8b615a7f48b
