raco setup: --tidy mode (as needed by `raco pkg remove') and more

When a collection disappears, then documentation and metadata
cross-references need fixing up based on everything that remains
available, even though other setup actions are confined to
collections that are specifically relevant to the packages. The
new `--tidy' mode takes care of that.

Package installation now also provides `--avoid-main' when working
with packages in a scope other than installation scope, and `raco
setup' now better respects `--avoid-main' to avoid creating
executables or re-running documentation.

Also, revise the `raco' documentation to better orient it toward the
package manager.
This commit is contained in:
Matthew Flatt 2013-04-17 10:08:27 -06:00
parent bcb05bc8cf
commit 51d48f3ab4
15 changed files with 432 additions and 241 deletions

View File

@ -130,7 +130,7 @@
[len (bytes-length skip-path)])
(and ((bytes-length b) . > . len)
(bytes=? (subbytes b 0 len) skip-path)))
(list -inf.0 "")))])
(cons -inf.0 "")))])
(let* ([sses (append
;; Find all .rkt/.ss/.scm files:
(filter extract-base-filename/ss (directory-list))

View File

@ -19,7 +19,9 @@
(if (list? s) s (list s)))
(append setup-collects
(if installation? '("scribblings/main") null)
'("scribblings/main/user")))))))
'("scribblings/main/user"))))
#:tidy? #t
#:avoid-main? (not installation?))))
(define ((pkg-error cmd) . args)
(apply raise-user-error

View File

@ -5,9 +5,11 @@
@title[#:tag "command"]{Adding a @exec{raco} Command}
The set of commands supported by @exec{raco} can be extended by
installed collections and @|PLaneT| packages. A command is added by
defining @indexed-racket[raco-commands] in the @filepath{info.rkt}
library of a collection or package (see @secref["info.rkt"]).
installed packages, @|PLaneT| packages, and other collections. A
command is added by defining @indexed-racket[raco-commands] in the
@filepath{info.rkt} library of a collection (see @secref["info.rkt"]),
and then @exec{raco setup} (as called directly or as part of a package
or @|PLaneT| installation) must index the @filepath{info.rkt} file.
The value bound to @racket[raco-commands] must be a list of command
specifications, where each specification is a list of four values:
@ -34,7 +36,7 @@ the command name used to load the command. When @exec{raco help} is
used on a command, the command is launched with an initial
@DFlag{help} argument in @racket[current-command-line-arguments].
The @racket[description-string] is a short string used to describe the
The @racket[_description-string] is a short string used to describe the
command in response to @exec{raco help}. The description should not be
capitalized or end with a period.
@ -47,7 +49,7 @@ more prominent. The @exec{pack} tool, which is currently ranked as the
least-prominent of the frequently used commands, has a value of
@racket[10].
As an example, the @filepath{info.rkt} of the "compiler" collection
As an example, the @filepath{info.rkt} of the @filepath{compiler} collection
might contain the
@racketblock[

View File

@ -12,6 +12,13 @@ The @exec{raco link} command inspects and modifies a @tech[#:doc
reference-doc]{collection links file} to display, add, or remove
mappings from collection names to filesystem directories.
Managing links directly is somewhat discouraged. Instead, use the
package manager (see @other-manual['(lib
"pkg/scribblings/pkg.scrbl")]), which installs and manages links
(i.e., it builds on @exec{raco link}) in a way that more gracefully
leads to sharing collections with others. Nevertheless, @exec{raco
link} is available for direct use.
For example, the command
@commandline{raco link maze}

View File

@ -4,6 +4,7 @@
racket/include
racket/contract
racket/future
racket/promise
compiler/cm
compiler/cm-accomplice
setup/parallel-build))

View File

@ -6,18 +6,22 @@
@title[#:tag "plt"]{@exec{raco pack}: Packing Library Collections}
@margin-note{Before creating a @filepath{.plt} archive to distribute,
consider instead posting your package on
@link["http://planet.racket-lang.org/"]{@|PLaneT|}.}
The @exec{raco pack} command creates an archive of files and
directories. Formerly, such archives were used directly to distribute
library files to Racket users, but the package manager (see
@other-manual['(lib "pkg/scribblings/pkg.scrbl")]) is now the
preferred mechanism for distribution.
The @exec{raco pack} command creates an archive for distributing
library files to Racket users. A distribution archive usually has the
suffix @as-index{@filepath{.plt}}, which DrRacket recognizes as an
archive to provide automatic unpacking facilities. The @exec{raco
setup} command (see @secref["setup"]) also supports @filepath{.plt}
unpacking with installation, while the @exec{raco unpack} command (see
@secref["unpack"]) unpacks an archive locally without attempting to
install it.
A packed archive usually has the suffix @as-index{@filepath{.plt}}.
The @exec{raco pkg} command recognizes a @filepath{.plt} archive for
installation as a package. The @exec{raco setup} command (see
@secref["setup"]) also supports @filepath{.plt} unpacking and
installation when using the @Flag{A} flag, but such installations do
not benefit from the more general management facilities of @exec{raco
pkg}, while the @exec{raco unpack} command (see @secref["unpack"])
unpacks an archive locally without attempting to install it. DrRacket
recognizes the @filepath{.plt} and currently treats such an archive in
the same way as @exec{raco setup -A}.
An archive contains the following elements:

View File

@ -46,85 +46,191 @@
@title[#:tag "setup" #:style 'toc]{@exec{raco setup}: Installation Management}
The @exec{raco setup} command finds, compiles, configures,
and installs documentation for all collections in a Racket
installation. It can also install single @filepath{.plt} files.
The @exec{raco setup} command builds bytecode, documentation,
executables, and metadata indexes for all installed collections.
The collections that are built by @exec{raco setup} can be part of the
original Racket distribution, installed via the package manager (see
@other-manual['(lib "pkg/scribblings/pkg.scrbl")]), installed via
@|PLaneT| (see @other-manual['(lib "planet/planet.scrbl")]), linked
via @exec{raco link}, in a directory that is listed in the
@envvar{PLTCOLLECTS} environment variable, or placed into one of the
default collection directories.
The @exec{raco setup} tool itself does not directly support the
installation of collections, except through the now-discouraged
@Flag{A} flag (see @secref["raco-setup-A"]). The @exec{raco setup} command is
used by installation tools such as the package manager or @|PLaneT|.
Programmers who modify installed collections may find it useful to run
@exec{raco setup} as an alternative to un-installing and re-installing
a set of collections.
@local-table-of-contents[]
@; ------------------------------------------------------------------------
@; ------------------------------------------------------------------------
@section[#:tag "running"]{Running @exec{raco setup}}
The @exec{raco setup} command performs two main services:
With no command-line arguments, @exec{raco setup} finds all of the
current collections---see @secref[#:doc ref-src]{collects}---and
compiles libraries in each collection. (Directories that are named
@filepath{.git} or @filepath{.svn} are not treated as collections.)
To restrict @exec{raco setup} to a set of collections, provide the
collection names as arguments. For example, @exec{raco setup
scribblings/raco} would only compile and render the documentation for
@exec{raco}, which is implemented in a @filepath{scribblings/raco}
collection.
An optional @filepath{info.rkt} within the collection can indicate
specifically how the collection's files are to be compiled and other
actions to take in setting up a collection, such as creating
executables or building documentation. See @secref["setup-info"] for
more information.
The @exec{raco setup} command accepts the following command-line
flags:
@itemize[
@item{@bold{Compiling and setting up all or some collections:}
When @exec{raco setup} is run without any arguments, it
finds all of the current collections---see @secref[#:doc
ref-src]{collects}---and compiles libraries in each collection.
(Directories that are named @filepath{.git} or @filepath{.svn} are
not treated as collections.)
@item{@DFlag{workers} @nonterm{n} or @Flag{j} @nonterm{n} --- use up
to @nonterm{n} parallel processes. By default, @exec{raco setup}
uses @racket[(processor-count)] processors, which typically uses
all of the machine's processing cores.}
To restrict @exec{raco setup} to a set of collections, provide the
collection names as arguments. For example, @exec{raco setup
scribblings/raco} would only compile and render the documentation
for @exec{raco}, which is implemented in a
@filepath{scribblings/raco} collection.
@item{@DFlag{only} --- restrict setup to specified collections and
@|PLaneT| packages, even if none are specified. This mode is the
default if any collection is specified as a command-line argument,
through the @Flag{l} flag, or through the @Flag{P} flag.}
An optional @filepath{info.rkt} within the collection can indicate
specifically how the collection's files are to be compiled and
other actions to take in setting up a collection, such as creating
executables or building documentation. See @secref["setup-info"]
for more information.
@item{@Flag{P} @nonterm{owner} @nonterm{package-name} @nonterm{maj}
@nonterm{min} --- constrain setup actions to the specified @|PLaneT|
package, in addition to any other specified @|PLaneT| packages or
@nonterm{collections}.}
The @DFlag{clean} (or @Flag{c}) flag to @exec{raco setup} causes it to
delete existing @filepath{.zo} files, thus ensuring a clean build
from the source files. The exact set of deleted files can be
controlled by @filepath{info.rkt}; see
@elemref["clean"]{@racket[clean]} for more information.
@item{@DFlag{tidy} --- remove metadata cache information and
documentation for non-existent collections (to clean up after removal)
even when setup actions are otherwise confined to specified collections.}
The @DFlag{workers} (or @Flag{j}) flag to @exec{raco setup} takes
an argument @racket[_n] to make compilation use up to @racket[_n]
parallel processes. The default value of @racket[_n] is
@racket[(processor-count)], which typically uses all the machine's
processing cores.
@item{@DFlag{clean} or @Flag{c} --- delete existing @filepath{.zo}
files, thus ensuring a clean build from the source files. The exact
set of deleted files can be controlled by @filepath{info.rkt}; see
@elemref["clean"]{@racket[clean]} for more information.}
The @DFlag{mode} @nonterm{mode} flag causes @exec{raco setup} to use a
@filepath{.zo} compiler other than the default compiler, and to put
the resulting @filepath{.zo} files in a subdirectory (of the usual
place) named by @nonterm{mode}. The compiler is obtained by using
@nonterm{mode} as a collection name, finding a
@filepath{zo-compile.rkt} module in that collection, and extracting
its @racket[zo-compile] export. The @racket[zo-compile] export
should be a function like @racket[compile]; see the
@filepath{errortrace} collection for an example.
@item{@DFlag{no-zo} or @Flag{n} --- refrain from compiling source
files to @filepath{.zo} files.}
When building @exec{racket}, flags can be provided to @exec{raco
setup} as run by @exec{make install} by setting the
@as-index{@envvar{PLT_SETUP_OPTIONS}} environment variable. For
example, the following command line uses a single process to build
collections during an install:
@item{@DFlag{trust-zos} --- fix timestamps on @filepath{.zo} files on
the assumption that they are already up-to-date.}
@commandline{env PLT_SETUP_OPTIONS="-j 1" make install}}
@item{@DFlag{no-launcher} or @Flag{x} --- refrain from creating
executables (as specified in @filepath{info.rkt} files; see
@secref["setup-info"]).}
@item{@bold{Unpacking @filepath{.plt} files:} A
@filepath{.plt} file is a platform-independent distribution archive
for software based on Racket. When one or more file names are
provided as the command line arguments to @exec{raco setup} with the @Flag{A} flag, the files
contained in the @filepath{.plt} archive are unpacked (according to
specifications embedded in the @filepath{.plt} file) and only
collections specified by the @filepath{.plt} file are compiled and
setup.}]
@item{@DFlag{no-install} or @Flag{i} --- refrain from running
pre-install actions (as specified in @filepath{info.rkt} files; see
@secref["setup-info"]).}
Run @exec{raco help setup} to see a list of all options accepted by
the @exec{raco setup} command.
@item{@DFlag{no-post-install} or @Flag{I} --- refrain from running
post-install actions (as specified in @filepath{info.rkt} files; see
@secref["setup-info"]).}
@item{@DFlag{no-info-domain} or @Flag{d} --- refrain from building
a cache of metadata information from @filepath{info.rkt}
files. This cache is needed by other tools. For example,
@exec{raco} itself uses the cache to locate plug-in tools.}
@item{@DFlag{no-docs} or @Flag{D} --- refrain from building
documentation.}
@item{@DFlag{doc-pdf} @nonterm{dir} --- in addition to building HTML
documentation, render documentation to PDF and place files in
@nonterm{dir}.}
@item{@DFlag{no-user} or @Flag{U} --- refrain from any user-specific
(as opposed to installation-specific) setup actions.}
@item{@DFlag{no-planet} --- refrain from any setup actions for
@|PLaneT| actions; this flags is implied by @DFlag{no-user}.}
@item{@DFlag{avoid-main} --- refrain from any setup actions that
affect the installation, as opposed to user-specific actions.}
@item{@DFlag{mode} @nonterm{mode} --- use a @filepath{.zo} compiler
other than the default compiler, and put the resulting
@filepath{.zo} files in a subdirectory (of the usual place) named
by @nonterm{mode}. The compiler is obtained by using @nonterm{mode}
as a collection name, finding a @filepath{zo-compile.rkt} module in
that collection, and extracting its @racket[zo-compile] export. The
@racket[zo-compile] export should be a function like
@racket[compile]; see the @filepath{errortrace} collection for an
example.}
@item{@DFlag{verbose} or @Flag{v} --- More verboase output about
@exec{raco setup} actions.}
@item{@DFlag{make-verbose} or @Flag{m} --- More verboase output about
dependency checks.}
@item{@DFlag{compiler-verbose} or @Flag{r} --- Even more verbose
output about dependency checks and compilation.}
@item{@DFlag{pause} or @Flag{p} --- Pause for user input if any
errors are reported (so that a user has time to inspect output that
might otherwise disappear when the @exec{raco setup} process ends).}
@item{@Flag{l} @nonterm{collection} @racket[...] --- constrain setup
actions to the specified @nonterm{collection}s (i.e., the same as
providing @nonterm{collections}s without a flag, but with no
possibility that a @nonterm{collection} is interpreted as a flag.}
@item{@Flag{A} @nonterm{archive} @racket[...] --- Install each
@nonterm{archive}; see @secref["raco-setup-A"].}
@item{@DFlag{force} --- for use with @Flag{A}, treat version
mismatches for archives as mere warnings.}
@item{@DFlag{all-users} or @Flag{a} --- for use with @Flag{A},
install archive into the installation instead of a user-specific
location.}
]
When building @exec{racket}, flags can be provided to @exec{raco
setup} as run by @exec{make install} by setting the
@as-index{@envvar{PLT_SETUP_OPTIONS}} environment variable. For
example, the following command line uses a single process to build
collections during an install:
@commandline{env PLT_SETUP_OPTIONS="-j 1" make install}
@; ------------------------------------------------------------------------
@subsection[#:tag "setup-info"]{Controlling @exec{raco setup} with @filepath{info.rkt} Files}
@section[#:tag "raco-setup-A"]{Installing @filepath{.plt} Archives}
A @filepath{.plt} file is a platform-independent distribution archive
for software based on Racket. A typical @filepath{.plt} file can be
installed as a package using @exec{raco pkg} (see @other-manual['(lib
"pkg/scribblings/pkg.scrbl")]), in which case @exec{raco pkg} supplies
facilities for uninstalling the package and managing dependencies.
An older approach is to supply a @filepath{.plt} file to @exec{raco
setup} with the @Flag{A} flag, the files contained in the
@filepath{.plt} archive are unpacked (according to specifications
embedded in the @filepath{.plt} file) and only collections specified
by the @filepath{.plt} file are compiled and setup. Archives processed
in this way can include arbitrary code that is executed at install
time, in addition to any actions triggered by the normal
collection-setup part of @exec{raco setup}.
Finally, the @exec{raco unpack} (see @secref["unpack"]) command can
list the content of a @filepath{.plt} archive or unpack the archive
without installing it as a package or collection.
@; ------------------------------------------------------------------------
@section[#:tag "setup-info"]{Controlling @exec{raco setup} with @filepath{info.rkt} Files}
To compile a collection's files to bytecode, @exec{raco setup} uses the
@racket[compile-collection-zos] procedure. That procedure, in turn,
@ -410,7 +516,7 @@ Optional @filepath{info.rkt} fields trigger additional actions by
@; ------------------------------------------------------------------------
@section[#:tag "setup-plt-plt"]{API for Installation}
@section[#:tag "setup-plt-plt"]{API for Setup}
@defmodule[setup/setup]
@ -425,6 +531,8 @@ Optional @filepath{info.rkt} fields trigger additional actions by
[#:make-user? make-user? any/c #t]
[#:make-docs? make-docs? any/c #t]
[#:clean? clean? any/c #f]
[#:tidy? tidy? any/c #f]
[#:avoid-main? avoid-main? any/c #f]
[#:jobs jobs exact-nonnegative-integer? #f]
[#:get-target-dir get-target-dir (or/c #f (-> path-string?)) #f])
void?]{
@ -449,6 +557,13 @@ Runs @exec{raco setup} with various options:
@item{@racket[clean?] --- if true, enables cleaning mode instead of setup mode}
@item{@racket[tidy?] --- if true, enables global tidying of
documentation and metadata indexes even when @racket[collections]
or @racket[planet-specs] is non-@racket[#f]}
@item{@racket[avoid-main?] --- if true, avoids setup actions that affect
the main installation, as opposed to user directories}
@item{@racket[jobs] --- if not @racket[#f], determines the maximum number of parallel
tasks used for setup}

View File

@ -9,7 +9,9 @@
The @exec{raco unpack} command unpacks a @filepath{.plt} archive (see
@secref["plt"]) to the current directory without attempting to install
any collections. Use @exec{raco setup -A} (see @secref["setup"]) to
any collections. Use @exec{raco pkg} (see @other-manual['(lib
"pkg/scribblings/pkg.scrbl")]) to install a @filepath{.plt} archive as
a package, or use @exec{raco setup -A} (see @secref["setup"]) to
unpack and install collections from a @filepath{.plt} archive.
Command-line flags:

View File

@ -3,5 +3,5 @@
(provide (struct-out cc))
(define-struct cc
(collection path name info omit-root info-root info-path info-path-mode shadowing-policy)
(collection path name info omit-root info-root info-path info-path-mode shadowing-policy main?)
#:inspector #f)

View File

@ -11,6 +11,7 @@
compiler-verbose
clean
compile-mode
make-only
make-zo
make-info-domain
make-launchers
@ -18,6 +19,7 @@
make-user
make-planet
avoid-main-installation
make-tidy
call-install
call-post-install
pause-on-errors

View File

@ -36,6 +36,7 @@
(define-flag-param compiler-verbose #f)
(define-flag-param clean #f)
(define-flag-param compile-mode #f)
(define-flag-param make-only #f)
(define-flag-param make-zo #t)
(define-flag-param make-launchers #t)
(define-flag-param make-info-domain #t)
@ -43,6 +44,7 @@
(define-flag-param make-user #t)
(define-flag-param make-planet #t)
(define-flag-param avoid-main-installation #f)
(define-flag-param make-tidy #f)
(define-flag-param call-install #t)
(define-flag-param call-post-install #t)
(define-flag-param pause-on-errors #f)

View File

@ -118,6 +118,8 @@
auto-start-doc? ; if #t, expands `only-dir' with [user-]start to
; catch new docs
make-user? ; are we making user stuff?
tidy? ; clean up, even beyond `only-dirs'
avoid-main? ; avoid main collection, even for `tidy?'
with-record-error ; catch & record exceptions
setup-printf)
(unless (doc-db-available?)
@ -191,7 +193,9 @@
(filter-user-docs (append-map (get-docs main-dirs) infos recs) make-user?)))
(define-values (main-docs user-docs) (partition doc-under-main? docs))
(unless only-dirs
(when (and (or (not only-dirs) tidy?)
(not avoid-main?)
(not latex-dest))
;; Check for extra document directories that we should remove
;; in the main installation:
(log-setup-info "checking installation document directories")
@ -300,28 +304,36 @@
(hash-set! out-path->info-cache path i)
i)))))
(define (tidy-database)
(when (and (or (not only-dirs) tidy?)
(not latex-dest))
(log-setup-info "tidying database")
(define files (make-hash))
(define tidy-docs (if tidy?
docs
(map info-doc infos)))
(define (get-files! main?)
(for ([doc (in-list tidy-docs)]
#:when (eq? main? (main-doc? doc)))
(hash-set! files (sxref-path latex-dest doc "in.sxref") #t)
(for ([c (in-range (add1 (doc-out-count doc)))])
(hash-set! files (sxref-path latex-dest doc (format "out~a.sxref" c)) #t))))
(unless avoid-main?
(get-files! #t)
(doc-db-clean-files main-db files))
(when (and (file-exists? user-db)
(not (equal? main-db user-db)))
(get-files! #f)
(doc-db-clean-files user-db files))))
(define main-db (find-doc-db-path latex-dest #f))
(define user-db (find-doc-db-path latex-dest #t))
(define (make-loop first? iter)
(let ([infos (filter-not info-failed? infos)]
[src->info (make-hash)]
[out-path->info-cache (make-hash)]
[main-db (find-doc-db-path latex-dest #f)]
[user-db (find-doc-db-path latex-dest #t)])
(unless only-dirs
(log-setup-info "cleaning database")
(define files (make-hash))
(define (get-files! main?)
(for ([i (in-list infos)]
#:when (eq? main? (main-doc? (info-doc i))))
(define doc (info-doc i))
(hash-set! files (sxref-path latex-dest doc "in.sxref") #t)
(for ([c (in-range (add1 (doc-out-count doc)))])
(hash-set! files (sxref-path latex-dest doc (format "out~a.sxref" c)) #t))))
(get-files! #t)
(doc-db-clean-files main-db files)
(when (and (file-exists? user-db)
(not (equal? main-db user-db)))
(get-files! #f)
(doc-db-clean-files user-db files)))
[out-path->info-cache (make-hash)])
(tidy-database)
;; Check for duplicate definitions
(when first?
(log-setup-info "checking for duplicates")
@ -532,6 +544,10 @@
(add1 count)))
(make-loop #f (add1 iter))))))
(unless infos
;; since we won't use `make-loop':
(tidy-database))
(when infos
(make-loop #t 0)
;; cache info to disk

View File

@ -33,6 +33,18 @@
#:program long-name
#:argv argv
#:once-each
[("-j" "--workers") workers "Use <#> parallel-workers"
(add-flags `((parallel-workers ,(string->number workers))))]
[("--only") "Set up only specified <collection>s"
(add-flags '((make-only #t)))]
#:multi
[("-P") owner package-name maj min
"Setup specified PLaneT packages only"
(set! x-specific-planet-packages (cons (list owner package-name maj min)
x-specific-planet-packages))]
#:once-each
[("--tidy") "Clear references to removed, even if not a specified <collection>"
(add-flags '((make-tidy #t)))]
[("-c" "--clean") "Delete existing compiled files; implies -nxi"
(add-flags '((clean #t)
(make-zo #f)
@ -40,10 +52,10 @@
(make-launchers #f)
(make-info-domain #f)
(make-docs #f)))]
[("-j" "--workers") workers "Use <#> parallel-workers"
(add-flags `((parallel-workers ,(string->number workers))))]
[("-n" "--no-zo") "Do not produce .zo files"
(add-flags '((make-zo #f)))]
[("--trust-zos") "Trust existing .zos (use only with prepackaged .zos)"
(add-flags '((trust-existing-zos #t)))]
[("-x" "--no-launcher") "Do not produce launcher programs"
(add-flags '((make-launchers #f)))]
[("-i" "--no-install") "Do not call collection-specific pre-installers"
@ -54,12 +66,16 @@
(add-flags '((make-info-domain #f)))]
[("-D" "--no-docs") "Do not compile .scrbl files and do not build documentation"
(add-flags '((make-docs #f)))]
[("--doc-pdf") dir "Build doc PDFs, write to <dir>"
(add-flags `((doc-pdf-dest ,dir)))]
[("-U" "--no-user") "Do not setup user-specific collections (implies --no-planet)"
(add-flags '((make-user #f) (make-planet #f)))]
[("--no-planet") "Do not setup PLaneT packages"
(add-flags '((make-planet #f)))]
[("--avoid-main") "Do not make main-installation files"
(add-flags '((avoid-main-installation #t)))]
[("--mode") mode "Select a compilation mode"
(add-flags `((compile-mode ,mode)))]
[("-v" "--verbose") "See names of compiled files and info printfs"
(add-flags '((verbose #t)))]
[("-m" "--make-verbose") "See make and compiler usual messages"
@ -67,18 +83,8 @@
[("-r" "--compile-verbose") "See make and compiler verbose messages"
(add-flags '((make-verbose #t)
(compiler-verbose #t)))]
[("--trust-zos") "Trust existing .zos (use only with prepackaged .zos)"
(add-flags '((trust-existing-zos #t)))]
[("-p" "--pause") "Pause at the end if there are any errors"
(add-flags '((pause-on-errors #t)))]
[("--force") "Treat version mismatches for archives as mere warnings"
(add-flags '((force-unpacks #t)))]
[("-a" "--all-users") "Install archives to main (not user-specific) installation"
(add-flags '((all-users #t)))]
[("--mode") mode "Select a compilation mode"
(add-flags `((compile-mode ,mode)))]
[("--doc-pdf") dir "Build doc PDFs, write to <dir>"
(add-flags `((doc-pdf-dest ,dir)))]
[("-l") => (lambda (flag . collections)
(check-collections short-name collections)
(cons 'collections (map list collections)))
@ -86,11 +92,11 @@
[("-A") => (λ (flag . archives)
(cons 'archives archives))
'("Unpack and install <archive>s" "archive")]
#:multi
[("-P") owner package-name maj min
"Setup specified PLaneT packages only"
(set! x-specific-planet-packages (cons (list owner package-name maj min)
x-specific-planet-packages))]
[("--force") "Treat version mismatches for archives as mere warnings"
(add-flags '((force-unpacks #t)))]
[("-a" "--all-users") "Install archives to main (not user-specific) installation"
(add-flags '((all-users #t)))]
#:handlers
(lambda (collections/archives . rest)
(let ([pre-archives (if (and (pair? collections/archives)

View File

@ -165,13 +165,17 @@
(if (make-planet) (specific-planet-dirs) null))
(define no-specific-collections?
(and (null? x-specific-collections) (null? x-specific-planet-dirs)))
(and (null? x-specific-collections)
(null? x-specific-planet-dirs)
(not (make-only))))
;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;; Find Collections ;;
;; ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
(define (make-cc* collection path omit-root info-root info-path info-path-mode shadowing-policy)
(define (make-cc* collection path omit-root info-root
info-path info-path-mode shadowing-policy
main?)
(define info
(or (with-handlers ([exn:fail? (warning-handler #f)]) (getinfo path))
(lambda (flag mk-default) (mk-default))))
@ -197,7 +201,8 @@
info
omit-root
info-root info-path info-path-mode
shadowing-policy)))
shadowing-policy
main?)))
(define ((warning-handler v) exn)
(setup-printf "WARNING" "~a" (exn->string exn))
@ -212,7 +217,8 @@
#:omit-root [omit-root #f]
#:info-root [given-info-root #f]
#:info-path [info-path #f]
#:info-path-mode [info-path-mode 'relative])
#:info-path-mode [info-path-mode 'relative]
#:main? [main? #f])
(define info-root
(or given-info-root
(ormap (lambda (p)
@ -243,7 +249,8 @@
info-path-mode
;; by convention, all collections have "version" 1 0. This
;; forces them to conflict with each other.
(list (cons 'lib (map path->string collection-p)) 1 0)))
(list (cons 'lib (map path->string collection-p)) 1 0)
main?))
(when new-cc
(hash-update! collection-ccs-table
collection-p
@ -281,7 +288,8 @@
#f ; don't need info-root; absolute paths in cache.rktd will be ok
(get-planet-cache-path)
'abs
(list `(planet ,owner ,pkg-file ,@extra-path) maj min))))
(list `(planet ,owner ,pkg-file ,@extra-path) maj min)
#f)))
;; planet-cc->sub-cc : cc (listof bytes [encoded path]) -> cc
;; builds a compilation job for the given subdirectory of the given cc this
@ -309,14 +317,16 @@
#:unless (skip-collection-directory? collection)
#:when (directory-exists? (build-path cp collection)))
(collection-cc! (list collection)
#:path (build-path cp collection)))
#:path (build-path cp collection)
#:main? (equal? cp (find-collects-dir))))
(let ([main-collects (find-collects-dir)])
(define (cc! col #:path path)
(collection-cc! col
#:path path
#:info-root main-collects
#:info-path-mode 'abs-in-relative
#:omit-root 'dir))
#:omit-root 'dir
#:main? #t))
(for ([c+p (in-list (links #:user? #f #:with-path? #t))])
(cc! (list (string->path (car c+p)))
#:path (cdr c+p)))
@ -371,23 +381,7 @@
(let loop ([l collections-to-compile])
(append-map (lambda (cc) (cons cc (loop (get-subs cc)))) l))))
(define (collection-tree-map collections-to-compile
#:skip-path [orig-skip-path (and (avoid-main-installation)
(find-collects-dir))])
(define skip-path
(and orig-skip-path
(path->bytes (simplify-path (if (string? orig-skip-path)
(string->path orig-skip-path)
orig-skip-path)
#f))))
(define (skip-path? path)
(and skip-path
(let ([b (path->bytes (simplify-path path #f))]
[len (bytes-length skip-path)])
(and ((bytes-length b) . > . len)
(bytes=? (subbytes b 0 len) skip-path)))
path))
(define (collection-tree-map collections-to-compile)
(define (build-collection-tree cc)
(define (make-child-cc parent-cc name)
(collection-cc! (append (cc-collection parent-cc) (list name))
@ -405,9 +399,7 @@
(if (eq? 'all omit)
(values null null)
(partition (lambda (x) (directory-exists? (build-path ccp x)))
(filter (lambda (p)
(not (or (member p omit)
(skip-path? p))))
(filter (lambda (p) (not (member p omit)))
(directory-list ccp)))))
(define children-ccs
(map build-collection-tree
@ -449,7 +441,8 @@
(define (check-against-all given-ccs nothing-else-to-do?)
(when (and (null? given-ccs)
nothing-else-to-do?)
nothing-else-to-do?
(not (make-tidy)))
(setup-printf #f "nothing to do")
(exit 1))
(define (cc->name cc)
@ -507,27 +500,32 @@
null))
(define top-level-plt-collects
(if no-specific-collections?
all-collections
(check-against-all
(append-map
(lambda (c)
(define elems
(append-map (lambda (s) (map string->path (regexp-split #rx"/" s)))
c))
(define ccs (collection->ccs elems))
(when (null? ccs)
;; let `collection-path' complain about the name, if that's the problem:
(with-handlers ([exn? (compose1 raise-user-error exn-message)])
(apply collection-path elems))
;; otherwise, it's probably a collection with nothing to compile
;; spell the name
(setup-printf "WARNING"
"nothing to compile in a given collection path: \"~a\""
(string-join c "/")))
ccs)
x-specific-collections)
(null? planet-collects))))
((if (avoid-main-installation)
(lambda (l) (filter (lambda (cc) (not (cc-main? cc))) l))
values)
(if no-specific-collections?
all-collections
(check-against-all
(append-map
(lambda (c)
(define sc (map (lambda (s) (if (path? s) (path->string s) s))
c))
(define elems
(append-map (lambda (s) (map string->path (regexp-split #rx"/" s)))
sc))
(define ccs (collection->ccs elems))
(when (null? ccs)
;; let `collection-path' complain about the name, if that's the problem:
(with-handlers ([exn? (compose1 raise-user-error exn-message)])
(apply collection-path elems))
;; otherwise, it's probably a collection with nothing to compile
;; spell the name
(setup-printf "WARNING"
"nothing to compile in a given collection path: \"~a\""
(string-join sc "/")))
ccs)
x-specific-collections)
(null? planet-collects)))))
(define planet-dirs-to-compile
(sort-collections
@ -818,8 +816,10 @@
(setup-printf #f "--- compiling collections ---")
(if ((parallel-workers) . > . 1)
(begin
(for/fold ([gcs 0]) ([cc (in-list (collection->ccs (list (string->path "racket"))))])
(compile-cc cc 0))
(when (or no-specific-collections?
(member "racket" x-specific-collections))
(for/fold ([gcs 0]) ([cc (in-list (collection->ccs (list (string->path "racket"))))])
(compile-cc cc 0)))
(managed-compile-zo (collection-file-path "parallel-build-worker.rkt" "setup"))
(with-specified-mode
(lambda ()
@ -854,93 +854,100 @@
(define ht (make-hash))
(define ht-orig (make-hash))
(define roots (make-hash))
(for ([cc ccs-to-compile])
(define-values [path->info-relative info-relative->path]
(define (get-info-ht info-root info-path info-path-mode)
(define-values (path->info-relative info-relative->path)
(apply values
(hash-ref roots
(cc-info-root cc)
info-root
(lambda ()
(define-values [p-> ->p]
(if (cc-info-root cc)
(make-relativize (lambda () (cc-info-root cc))
(if info-root
(make-relativize (lambda () info-root)
'info
'path->info-relative
'info-relative->path)
(values #f #f)))
(hash-set! roots (cc-info-root cc) (list p-> ->p))
(hash-set! roots info-root (list p-> ->p))
(list p-> ->p)))))
(hash-ref ht info-path
(lambda ()
;; No table for this root, yet. Build one.
(define l
(let ([p info-path])
(if (file-exists? p)
(with-handlers ([exn:fail? (warning-handler null)])
(with-input-from-file p read))
null)))
;; Convert list to hash table. Include only well-formed
;; list elements, and only elements whose corresponding
;; collection exists.
(define t (make-hash))
(define all-ok? #f)
(when (list? l)
(set! all-ok? #t)
(for ([i l])
(match i
[(list (and a (or (? bytes?) (list 'info (? bytes?) ...)))
(list (? symbol? b) ...) c (? integer? d) (? integer? e))
(define p (if (bytes? a) (bytes->path a) a))
;; Check that the path is suitably absolute or relative:
(define dir
(case info-path-mode
[(relative abs-in-relative)
(or (and (list? p)
(info-relative->path p))
(and (complete-path? p)
;; `c' must be `(lib ...)'
(list? c)
(pair? c)
(eq? 'lib (car c))
(pair? (cdr c))
(andmap string? (cdr c))
;; Path must match collection resolution:
(with-handlers ([exn:fail? (lambda (exn) #f)])
(equal? p (apply collection-path (cdr c))))
p))]
[(abs)
(and (complete-path? p) p)]))
(if (and dir
(let ([omit-root
(if (path? p)
;; absolute path => need a root for checking omits;
;; for a collection path of length N, go up N-1 dirs:
(simplify-path (apply build-path p (for/list ([i (cddr c)]) 'up)) #f)
;; relative path => no root needed for checking omits:
#f)])
(and (directory-exists? dir)
(not (eq? 'all (omitted-paths dir getinfo omit-root)))))
(or (file-exists? (build-path dir "info.rkt"))
(file-exists? (build-path dir "info.ss"))))
(hash-set! t a (list b c d e))
(begin (when (verbose) (printf " drop entry: ~s\n" i))
(set! all-ok? #f)))]
[_ (when (verbose) (printf " bad entry: ~s\n" i))
(set! all-ok? #f)])))
;; Record the table loaded for this collection root in the
;; all-roots table:
(hash-set! ht info-path t)
;; If anything in the "cache.rktd" file was bad, then claim
;; that the old table was empty, so that we definitely write
;; the new table.
(hash-set! ht-orig info-path
(and all-ok? (hash-copy t)))
t)))
;; process all collections:
(for ([cc ccs-to-compile])
(define domain
(with-handlers ([exn:fail? (lambda (x) (lambda () null))])
(dynamic-require (build-path (cc-path cc) "info.rkt")
'#%info-domain)))
;; Check whether we have a table for this cc's info-domain cache:
(define t
(hash-ref ht (cc-info-path cc)
(lambda ()
;; No table for this root, yet. Build one.
(define l
(let ([p (cc-info-path cc)])
(if (file-exists? p)
(with-handlers ([exn:fail? (warning-handler null)])
(with-input-from-file p read))
null)))
;; Convert list to hash table. Include only well-formed
;; list elements, and only elements whose corresponding
;; collection exists.
(define t (make-hash))
(define all-ok? #f)
(when (list? l)
(set! all-ok? #t)
(for ([i l])
(match i
[(list (and a (or (? bytes?) (list 'info (? bytes?) ...)))
(list (? symbol? b) ...) c (? integer? d) (? integer? e))
(define p (if (bytes? a) (bytes->path a) a))
;; Check that the path is suitably absolute or relative:
(define dir
(case (cc-info-path-mode cc)
[(relative abs-in-relative)
(or (and (list? p)
(info-relative->path p))
(and (complete-path? p)
;; `c' must be `(lib ...)'
(list? c)
(pair? c)
(eq? 'lib (car c))
(pair? (cdr c))
(andmap string? (cdr c))
;; Path must match collection resolution:
(with-handlers ([exn:fail? (lambda (exn) #f)])
(equal? p (apply collection-path (cdr c))))
p))]
[(abs)
(and (complete-path? p) p)]))
(if (and dir
(let ([omit-root
(if (path? p)
;; absolute path => need a root for checking omits;
;; for a collection path of length N, go up N-1 dirs:
(simplify-path (apply build-path p (for/list ([i (cddr c)]) 'up)) #f)
;; relative path => no root needed for checking omits:
#f)])
(and (directory-exists? dir)
(not (eq? 'all (omitted-paths dir getinfo omit-root)))))
(or (file-exists? (build-path dir "info.rkt"))
(file-exists? (build-path dir "info.ss"))))
(hash-set! t a (list b c d e))
(begin (when (verbose) (printf " drop entry: ~s\n" i))
(set! all-ok? #f)))]
[_ (when (verbose) (printf " bad entry: ~s\n" i))
(set! all-ok? #f)])))
;; Record the table loaded for this collection root in the
;; all-roots table:
(hash-set! ht (cc-info-path cc) t)
;; If anything in the "cache.rktd" file was bad, then claim
;; that the old table was empty, so that we definitely write
;; the new table.
(hash-set! ht-orig (cc-info-path cc)
(and all-ok? (hash-copy t)))
t)))
;; Get the table for this cc's info-domain cache:
(define t (get-info-ht (cc-info-root cc)
(cc-info-path cc)
(cc-info-path-mode cc)))
(define-values (path->info-relative info-relative->path)
;; Look up value that was forced by by `get-info-ht':
(apply values (hash-ref roots (cc-info-root cc))))
;; Add this collection's info to the table, replacing any information
;; already there, if the collection has an "info.ss" file:
(when (or (file-exists? (build-path (cc-path cc) "info.rkt"))
@ -954,6 +961,19 @@
;; Use absolute path:
(path->bytes (cc-path cc)))
(cons (domain) (cc-shadowing-policy cc)))))
;; In "tidy" mode, make sure we check each "cache.rktd":
(when (make-tidy)
(for ([c (in-list (current-library-collection-paths))])
(when (and (directory-exists? c)
(not (and (avoid-main-installation)
(equal? c (find-collects-dir)))))
(define info-path (build-path c "info-domain" "compiled" "cache.rktd"))
(when (file-exists? info-path)
(get-info-ht c info-path 'relative))))
(when (make-user)
(define info-path (get-planet-cache-path))
(when (file-exists? info-path)
(get-info-ht #f info-path 'abs))))
;; Write out each collection-root-specific table to a "cache.rktd" file:
(hash-for-each ht
(lambda (info-path ht)
@ -998,6 +1018,7 @@
name-str
(if no-specific-collections? #f (map cc-path ccs-to-compile))
latex-dest auto-start-doc? (make-user)
(make-tidy) (avoid-main-installation)
(lambda (what go alt) (record-error what "Building docs" go alt))
setup-printf))

View File

@ -22,6 +22,8 @@
#:make-docs? [make-docs? #t]
#:make-user? [make-user? #t]
#:clean? [clean? #f]
#:tidy? [tidy? #f]
#:avoid-main? [avoid-main? #f]
#:jobs [parallel #f])
(define-unit set-options@
(import setup-option^ compiler^)
@ -43,12 +45,21 @@
(when collections
(specific-collections collections))
(when (or planet-specs collections)
(make-only #t))
(unless make-user?
(make-user #f))
(unless make-docs?
(make-docs #f))
(when tidy?
(make-tidy #t))
(when avoid-main?
(avoid-main-installation #t))
(when clean?
(clean #t)
(make-zo #f)