diff --git a/pkgs/racket-build-guide/build.scrbl b/pkgs/racket-build-guide/build.scrbl new file mode 100644 index 0000000000..ed91be9ffe --- /dev/null +++ b/pkgs/racket-build-guide/build.scrbl @@ -0,0 +1,342 @@ +#lang scribble/manual +@(require "common.rkt" + scribble/bnf) + +@title[#:tag "build"]{Building Racket from Source} + +In a checkout of the Racket @hyperlink[git-repo]{Git repository}, you +could try just running + +@commandline{make} + +but we recommend that you at least consider the information in +@secref["src"] and @secref["modes"]. + +@; ------------------------------------------------------------ +@section[#:tag "src"]{Git Repository versus Source Distribution} + +Instead of building from the @hyperlink[git-repo]{Git repository}, +consider getting source for the current Racket release from + +@centerline{@url{http://download.racket-lang.org/}} + +or get a source snapshot (updated daily) from + +@centerline{@url{http://snapshot.racket-lang.org/}} + +The @onscreen{Source + built packages} options from those sites will +build and install especially quickly, because platform-independent +bytecode and documentation are pre-built. + +In contrast to the Git repository, release and snapshot source +distributions will work in the + +@commandline{configure --prefix=... && make && make install} + +way that you probably expect. + +@; ------------------------------------------------------------ +@section[#:tag "modes"]{Git Repository Build Modes} + +The rest of this chapter assumes that you're sticking with the +@hyperlink[git-repo]{source repository}. In that case, you still have +several options: + +@itemlist[ + + @item{@bold{In-place build} --- This mode is the default. It creates + a build in the @filepath{racket} subdirectory and installs packages + that you specify (or the @filepath{main-distribution} plus + @filepath{main-distribution-test} package by default). Any package + implementations that reside in the @filepath{pkgs} subdirectory are + linked in-place. This is the most natural mode for developing + Racket itself or staying on the bleeding edge. See + @secref["quick-in-place"] for more instructions.} + + @item{@bold{Unix-style install} --- This mode installs to a given + destination directory (on platforms other Windows), leaving no + reference to the source directory. This is the most natural mode + for installing once from the source repository. See + @secref["quick-unix-style"] for more instructions.} + + @item{@bold{Minimal} --- This mode is like a source distribution, and + it is described in the @filepath{src} subdirectory of + @filepath{racket} (i.e., ignore the repository's root directory and + @filepath{pkgs} subdirectory). Build a minimal Racket using the + usual @exec{configure && make && make install} steps (or similar + for Windows), and then you can install packages from the catalog + server with @exec{raco pkg}.} + + @item{@bold{Installers} --- This mode creates Racket distribution + installers for a variety of platforms by farming out work to + machines that run those platforms. This is the way that Racket + snapshots and releases are created, and you can create your own. + See @secref["distribute"] for more instructions.} + + @item{@bold{In-place Racket on Chez Scheme build} --- This mode + builds using Chez Scheme via @exec{make cs}. Unless you use various + options described in @secref["build-cs"], this process downloads + Chez Scheme from GitHub, builds a traditional @exec{racket} with + minimal packages, builds Chez Scheme, and then builds Racket on + Chez Scheme using Racket and Chez Scheme. Final executables with + names that end in @litchar{cs} or @litchar{CS} are the Racket on + Chez Scheme variants.} + +] + +@; ------------------------------------------------------------ +@section[#:tag "quick-in-place"]{Quick Instructions: In-Place Build} + +On Unix (including Linux) and Mac OS, @exec{make} (or @exec{make in-place}) +creates a build in the @filepath{racket} directory. + +On Windows with Microsoft Visual Studio (any version between 2008/9.0 +and 2019/16.0), @exec{nmake win32-in-place} creates a build in the +@filepath{racket} directory. For information on configuring your +command-line environment for Visual Studio, see +@filepath{racket/src/worksp/README.txt}. + +On Windows with MinGW, use @exec{make PLAIN_RACKET=racket/racket}, +since MinGW uses Unix-style tools but generates a Windows-layout +Racket build. + +In all cases, an in-place build includes (via links) a few packages +that are in the @filepath{pkgs} directory. To get new versions of +those packages, as well as the Racket core, then use @exec{git pull}. +Afterward, or to get new versions of any other package, use @exec{make +in-place} again, which includes a @exec{raco pkg update} step. + +See @secref["more"] for more information. + + +@; ------------------------------------------------------------ +@section[#:tag "quick-unix-style"]{Quick Instructions: Unix-Style Install} + +On Unix (including Linux), @exec{make unix-style PREFIX=@nonterm{dir}} +builds and installs into @filepath{@nonterm{dir}} (which must be an +absolute path) with binaries in @filepath{@nonterm{dir}/bin}, packages +in @filepath{@nonterm{dir}/share/racket/pkgs}, documentation in +@filepath{@nonterm{dir}/share/racket/doc}, etc. + +On Mac OS, @exec{make unix-style PREFIX=@nonterm{dir}} builds and +installs into @filepath{@nonterm{dir}} (which must be an absolute +path) with binaries in @filepath{@nonterm{dir}/bin}, packages in +@filepath{@nonterm{dir}/share/pkgs}, documentation in +@filepath{@nonterm{dir}/doc}, etc. + +On Windows, Unix-style install is not supported. + +A Unix-style install leaves no reference to the source directory. + +To split the build and install steps of a Unix-style installation, +supply @exec{DESTDIR=@nonterm{dest-dir}} with @exec{make unix-style +PREFIX=@nonterm{dir}}, which assembles the installation in +@filepath{@nonterm{dest-dir}} (which must be an absolute path). Then, +copy the content of @filepath{@nonterm{dest-dir}} to the target root +@filepath{@nonterm{dir}}. + +See @secref["more"] for more information. + +@; ------------------------------------------------------------ +@section[#:tag "more"]{More Instructions: Building Racket} + +The @filepath{racket} directory contains minimal Racket, which is just +enough to run @exec{raco pkg} to install everything else. The first +step of @exec{make in-place} or @exec{make unix-style} is to build +minimal Racket, and you can read @filepath{racket/src/README} for more +information. + +If you would like to provide arguments to @exec{configure} for the +minimal Racket build, then you can supply them with by adding +@exec{CONFIGURE_ARGS_qq="@nonterm{options}"} to @exec{make in-place} +or @exec{make unix-style}. (The @tt{_qq} suffix on the variable name +@tt{CONFIGURE_ARGS_qq} is a convention that indicates that single- and +double-quote marks are allowed in the value.) + +The @filepath{pkgs} directory contains packages that are tied to the +Racket core implementation and are therefore kept in the same Git +repository. A @exec{make in-place} links to the package in-place, +while @exec{make unix-style} copies packages out of @filepath{pkgs} to +install them. + +To install a subset of the packages in @filepath{pkgs}, supply @exec{PKGS} value to +@exec{make}. For example, + +@commandline{make PKGS="gui-lib readline-lib} + +links only the @filepath{gui-lib} and @filepath{readline-lib} packages +and their dependencies. The default value of @exec{PKGS} is +@tt{"main-distribution main-distribution-test"}. If you run @tt{make} +a second time, all previously installed packages remain installed and +are updated, while new packages are added. To uninstall previously +selected package, use @exec{raco pkg remove}. + +To build anything other than the latest sources in the repository +(e.g., when building from the @tt{v6.2.1} tag), you need a catalog +that's compatible with those sources. Note that a release distribution +is configured to use a catalog specific to that release, so you can +extract the catalog's URL from there. + +Using @exec{make} (or @exec{make in-place}) sets the installation's +name to @tt{development}, unless the installation has been previously +configured (i.e., unless the @filepath{racket/etc/config.rktd} file +exists). The installation name affects, for example, the directory +where user-specific documentation is installed. Using @exec{make} also +sets the default package scope to @exec{installation}, which means +that packages are installed by default into the installation's space +instead of user-specific space. The name and/or default-scope +configuration can be changed through @exec{raco pkg config}. + +Note that @exec{make -j @nonterm{n}} controls parallelism for the +makefile part of a build, but not for the @exec{raco setup} part. To +control both the makefile and the @exec{raco setup} part, use + +@commandline{make CPUS=@nonterm{n}} + +which recurs with @exec{make -j JOB_OPTIONS="-j "}. Setting +@exec{CPUS} also works with @exec{make unix-style}. + +Use @exec{make as-is} (or @exec{nmake win32-as-is}) to perform the +same build actions as @exec{make in-place}, but without consulting any +package catalogs or package sources to install or update packages. In +other words, use @exec{make as-is} to rebuild after local changes that +could include changes to the Racket core. (If you change only +packages, then @exec{raco setup} should suffice.) + +If you need even more control over the build, carry on to +@secref["even-more"] further below. + + +@; ------------------------------------------------------------ +@section[#:tag "build-cs"]{More Instructions: Building Racket on Chez Scheme} + +The @exec{make cs} target (or @exec{make cs-as-is} for a rebuild, or +@exec{nmake win32-cs} on Windows with Visual Studio) builds a variant +of Racket that runs on Chez Scheme. By default, the executables for +the Racket-on-Chez variant all have a @litchar{cs} or @litchar{CS} +suffix, and they coexist with a traditional Racket build by keeping +compiled files in a machine-specific subdirectory of the +@filepath{compiled} directory. You can remove the @litchar{cs} suffix +and the subdirectory in @filepath{compiled} by providing +@exec{RACKETCS_SUFFIX=""} to @exec{make}. (One day, if all goes well, +the default for @exec{RACKETCS_SUFFIX} will change from @tt{"cs"} to +@tt{""}.) + +Building Racket on Chez Scheme requires an existing Racket and Chez +Scheme. If you use @exec{make cs} with no further arguments, then the +build process will bootstrap by building a traditional variant of +Racket and by downloading and building Chez Scheme. + +If you have a sufficiently recent Racket installation already with at +least the @filepath{compiler-lib} package installed, you can supply +@exec{RACKET=...} with @exec{make cs} to skip that part of the +bootstrap. And if you have a Chez Scheme source directory already, you +can supply that with @exec{SCHEME_SRC=@nonterm{dir}} instead of +downloading a new copy: + +@margin-note{For now, Racket on Chez requires the variant of Chez Scheme at + @url{https://github.com/racket/ChezScheme}} + +@commandline{make cs RACKET=racket SCHEME_SRC=path/to/ChezScheme} + +Use @exec{make both} to build both traditional Racket and Racket on +Chez Scheme, where packages are updated documentation is built only +once (using traditional Racket). + + +@; ------------------------------------------------------------ +@section[#:tag "even-more"]{Even More Instructions: Building Racket Pieces} + +Instead of just using @exec{make in-place} or @exec{make unix-style}, you can +take more control over the build by understanding how the pieces fit +together. + +@; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +@subsection{Building Minimal Racket} + +Instead of using the top-level makefile, you can go into +@filepath{racket/src} and follow the @filepath{README.txt} there, +which gives you more configuration options. + +If you don't want any special configuration and you just want the base +build, you can use @exec{make base} (or @exec{nmake win32-base}) with the +top-level makefile. + +Minimal Racket does not require additional native libraries to run, +but under Windows, encoding-conversion, extflonum, and SSL +functionality is hobbled until native libraries from the +@filepath{racket-win32-i386} or @filepath{racket-win32-x86_64} package +are installed. + +On all platforms, from the top-level makefile, @exec{JOB_OPTIONS} as a +makefile variable and @exec{PLT_SETUP_OPTIONS} as an environment +variable are passed on to the @exec{raco setup} that is used to build +minimal-Racket libraries. See the documentation for @exec{raco setup} +for information on the options. + +For cross compilation, add configuration options to +@exec{CONFIGURE_ARGS_qq="@nonterm{options}"} as described in the +@filepath{README.txt} of @filepath{racket/src}, but also add a +@exec{PLAIN_RACKET=...} argument for the top-level makefile to specify +the same executable as in an @exec{--enable-racket=...} for +@exec{configure}. In general, the @exec{PLAIN_RACKET} setting should +have the form @exec{PLAIN_RACKET="@nonterm{exec} -C"} to ensure that +cross-compilation mode is used and that any foreign libraries needed +for build time can be found, but many cross-compilation scenarios work +without @Flag{C}. + +Specify @exec{SETUP_MACHINE_FLAGS=@nonterm{options}} to set Racket +flags that control the target machine of compiled bytecode for +@exec{raco setup} and @exec{raco pkg install}. For example +@exec{SETUP_MACHINE_FLAGS=-M} causes the generated bytecode to be +machine-independent, which is mainly useful when the generated +installation will be used as a template for other platforms or for +cross-compilation. + +@; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +@subsection{Installing Packages} + +After you've built and installed minimal Racket, you could install +packages via the package-catalog server, completely ignoring the +content of @filepath{pkgs}. + +If you want to install packages manually out of the @filepath{pkgs} +directory, the @exec{local-catalog} target creates a catalog as +@filepath{racket/local/catalog} that merges the currently configured +catalog's content with pointers to the packages in @filepath{pkgs}. A +Unix-style build works that way: it builds and installs minimal +Racket, and then it installs packages out of a catalog that is created +by @exec{make local-catalog}. + +To add a package catalog that is used after the content of +@filepath{pkgs} but before the default package catalogs, specify the +catalog's URL as the @exec{SRC_CATALOG} makefile variable: + +@commandline{make .... SRC_CATALOG=@nonterm{url}} + +@; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - +@subsection{Linking Packages for In-Place Development Mode} + +With an in-place build, you can edit packages within @filepath{pkgs} directly +or update those packages with @exec{git pull} plus @exec{raco setup}, since the +packages are installed with the equivalent of @exec{raco pkg install -i +--static-link @nonterm{path}}. + +Instead of actually using @exec{raco pkg install --static-link ...}, the +@exec{pkgs-catalog} makefile target creates a catalog that points to the +packages in @filepath{pkgs}, and the catalog indicates that the packages are to +be installed as links. The @exec{pkgs-catalog} target further configures +the new catalog as the first one to check when installing +packages. The configuration adjustment is made only if no +configuration file @filepath{racket/etc/config.rktd} exists already. + +All other packages (as specified by @exec{PKGS}) are installed via the +configured package catalog. They are installed in installation scope, but +the content of @filepath{racket/share/pkgs} is not meant to be edited. To +reinstall a package in a mode suitable for editing and manipulation +with Git tools, use + +@commandline{raco pkg update --clone extra-pkgs/@nonterm{pkg-name}} + +The @filepath{extra-pkgs} directory name is a convention that is supported by a +@filepath{.gitignore} entry in the repository root. diff --git a/pkgs/racket-build-guide/common.rkt b/pkgs/racket-build-guide/common.rkt new file mode 100644 index 0000000000..d54b69383c --- /dev/null +++ b/pkgs/racket-build-guide/common.rkt @@ -0,0 +1,5 @@ +#lang racket/base + +(provide git-repo) + +(define git-repo "https://github.com/racket/racket") diff --git a/pkgs/racket-build-guide/contribute.scrbl b/pkgs/racket-build-guide/contribute.scrbl new file mode 100644 index 0000000000..cd1492635a --- /dev/null +++ b/pkgs/racket-build-guide/contribute.scrbl @@ -0,0 +1,189 @@ +#lang scribble/manual +@(require "common.rkt" + scribble/bnf + (for-label (only-in scribble/manual history))) + +@title[#:tag "contribute"]{Contibuting to Racket Development} + +The Racket developers are happy to receive bug reports and +improvements to the implementation and documentation through GitHub +issues and pull requests: + +@itemlist[ + + @item{Issues (bug reports): @url{https://github.com/racket/racket/issues}} + + @item{Pull requests (improvements): @url{https://github.com/racket/racket/pulls}} + +] + +The Racket distribution includes scores of packages that have their +own separate repositories, which somewhat complicates the process of +sending pull requests. The mechanism is the same, but see +@secref["pkg-contribute"] for more guidance. + +By making a contribution, you are agreeing that your contribution is +licensed under the LGPLv3, Apache 2.0, and MIT licenses. Those +licenses are available in the @hyperlink[git-repo]{Racket Git +repository} in the files @filepath{LICENSE.txt}, +@filepath{LICENSE-APACHE.txt}, and @filepath{LICENSE-MIT.txt}. + +@; ------------------------------------------------------------ +@section[#:tag "main-contribute"]{Main-Repository Contributions} + +The @hyperlink[git-repo]{main Racket Git repository} contains the +implementation of everything that is in the Minimal Racket +distribution. That includes the runtime system, core libraries, and +@exec{raco pkg} so that other packages can be installed. + +The main Racket repository also has the source to the Racket +Reference, Racket Guide, and other core-ish documentation, including +the source to the document that you are reading. Those document +sources are in the repository's @filepath{pkgs} directory. + +Finally, the main repository includes a few other packages that are +especially tightly bound to the runtime-system implementation, such as +the @filepath{compiler-lib} package or the @filepath{racket-test} +package. Those package sources are also in the repository's +@filepath{pkgs} directory. + +To develop improvements to any of those parts of Racket, following the +usual GitHub-based workflow: + +@itemlist[ + + @item{Fork the Racket repository.} + + @item{Create an in-place build as described in @secref["build"].} + + @item{Make your changes and rebuild with @exec{make} or @exec{make + as-is} or @exec{raco setup}, where @exec{raco setup} is the + best choice when modifying Racket libraries that are in + @filepath{collects} or a package.} + + @item{Commit changes to your fork and + @hyperlink["https://help.github.com/en/articles/creating-a-pull-request"]{submit + a pull request}.} + +] + +See the @secref["contribute-guidelines"]. + +The variant of Chez Scheme that is needed to build Racket on Chez +Scheme has its own repository (to preserve the shape of the original +Chez Scheme reporitory): @url{https://github.com/racket/ChezScheme}. + +@; ------------------------------------------------------------ +@section[#:tag "pkg-contribute"]{Distribution-Package Contributions} + +If you find yourself changing a file that is in a +@filepath{share/pkgs} subdirectory, then that file is not part of the +main Racket Git repository. It almost certainly has its own Git +repository somewhere else, possibly within +@url{https://github.com/racket}, but possibly in another user's space. +The name of the directory in @filepath{share/pkgs} is almost certainly +the package name. + +To start working on a package @nonterm{pkg-name}, it's usually best to +go to the root directory of your Racket repository checkout and run + +@commandline{raco pkg update --clone extra-pkgs/@nonterm{pkg-name}} + +That will create @filepath{extra-pkgs/@nonterm{pkg-name}} as a clone +of the package's source Git repository, it will replace the current +installation of the package in your Racket build to point at that +directory, and then it will rebuild (essentially by using @exec{raco +setup}) with the new location of the package installation. Now you can +edit in @filepath{extra-pkgs/@nonterm{pkg-name}}, and your changes +will be live. + +Some information that might improve your experience: + +@itemlist[ + + @item{You can add @DFlag{no-setup} to the @exec{raco pkg update} + command to skip the @exec{raco setup} step, which makes sense + if you want to make changes and then run @exec{raco setup} + yourself.} + + @item{A package is sometimes a subdirectory within a Git respository, + and it would be better if the checkout in @filepath{extra-pkgs} + matched the respoitory name instead of the package name. If you + know the repository name, you can use + + @commandline{raco pkg update --clone extra-pkgs/@nonterm{repo-name} @nonterm{pkg-name}} + + to make the distinction.} + + @item{This same approach will generally work if you're starting from + a distribution installer instead of the checkout of the Raclet + sources from the main Git repository. You'll need write + permission to the installation, though, so that @exec{raco pkg + update} can redirect the package. Also, there's no particular + reason to use @exec{extra-pkgs} in that case.} + + @item{If you're done and want to go back to the normal installation + for @nonterm{pkg-name}, use + + @commandline{raco pkg update --catalog @nonterm{pkg-name}}} + + @item{See @secref["git-workflow" #:doc '(lib + "pkg/scribblings/pkg.scrbl")] for more information about how + packages are meant to work as Git repositories.} + +] + +Note that none of this is necessary if you're modifying a package in +the main Racket repository's @filepath{pkgs} directory. Those are +automatically linked in place for an in-place build of Racket. + +@; ------------------------------------------------------------ +@section[#:tag "contribute-guidelines"]{General Contribution Guidelines} + +When you make a pull request, the Racket developers will help you get +the improvement in shape to merge to the Racket repository. You can +make that process faster by keeping a few guidelines in mind: + +@itemlist[ + + @item{Try to follow the @seclink["top" #:doc '(lib "scribblings/style/style.scrbl")]{style guide}.} + + @item{When you fix a bug or create a new feature, include a test + case for it. + + Note that core Racket tests are in + @filepath{pkgs/racket-test-core/tests/racket}, and tests for + other libraries are also sometimes in a separate + @filepath{-test} package.} + + @item{Include new or updated documentation as appropriate. + + Note that the Racket reference is in + @filepath{pkgs/racket-doc/scribblings/reference}, and + documentation for other libraries are also sometimes in a + separate @filepath{-doc} package. + + When adding to a library or extending an existing binding's + behavior, be sure to include a @racket[history] note in the + documentation to record the change.} + + @item{Build with your changes. + + Don't break the Racket build. That means at least checking that + @exec{raco setup} runs and completes without errors. If you + added or modified documentation, visually inspect the newly + rendered documentation to make sure it reads as intended. + + A common mistake is to just run a modified library or its + tests, but where a change creates a new package dependency that + will only be detected by a full @exec{raco setup}. + @italic{Really:} run @exec{raco setup}.} + +] + +@; ------------------------------------------------------------ +@section[#:tag "contribute-more"]{More Resources} + +For additional pointers on how to contribute to Racket, see + +@centerline{@url{https://github.com/racket/racket/wiki/Ways-to-contribute-to-Racket}} diff --git a/pkgs/racket-build-guide/distribute.scrbl b/pkgs/racket-build-guide/distribute.scrbl new file mode 100644 index 0000000000..87de2e0d8e --- /dev/null +++ b/pkgs/racket-build-guide/distribute.scrbl @@ -0,0 +1,398 @@ +#lang scribble/manual +@(require "common.rkt" + scribble/bnf) + +@(define distro-build-doc '(lib "distro-build/distro-build.scrbl")) + +@(define distro-build-package + @seclink["top" + #:doc '(lib "distro-build/distro-build.scrbl")]{the @filepath{distro-build} package}) + +@title[#:tag "distribute"]{Distributing Racket Variants} + +This chapter is about distributing variants of Racket, as opposed to +distributing applications that are built with Racket. See +@secref["exe-dist" #:doc '(lib "scribblings/raco/raco.scrbl")] +for information about distributing applications. + +@bold{Important:} To build installers that can be distributed to other +users, do not use @exec{make in-place} or @exec{make unix-style}, but +instead start from a clean repository. + +Use one non-Windows machine as a server, where packages will be +pre-built. Then, as described below, create platform-specific +installers on some number of client machines, each of which contacts +the server machine to obtain pre-built packages. The server can act as +a client, naturally, to create an installer for the server's platform. + +GNU @exec{make} is required on the server machine, @exec{nmake} is +required on Windows client machines, and any @exec{make} should work +on other client machines. + +The distribution-build process is a collaboration between the Racket +Git repository's top-level makefile and @|distro-build-package|. + +@; ------------------------------------------------------------ +@section{Running Build Farms} + +The @exec{installers} target of the makefile will do everything to +generate installers: build a server on the current machine, run +clients on hosts specified via @exec{CONFIG}, and start/stop +VirtualBox virtual machines that act as client machines. + +If the server is already built, the @exec{installers-from-built} +target will drive the client builds without re-building the server. + +See the documentation of @|distro-build-package| for a description of +the site-configuration module and requirements on client hosts. + +If @filepath{my-site-config.rkt} is a configuration module, then + +@commandline{make installers CONFIG=my-site-config.rkt} + +drives the build farm, and the resulting installers are in +@filepath{build/installers}, with a hash table mapping descriptions to +installer filenames in @filepath{build/installer/table.rktd}. A log +file for each client is written to @filepath{build/log}. + +If you have the @filepath{distro-build-server} package installed in +some Racket build (not the one for building installers), you can use + +@commandline{make describe-clients CONFIG=my-site-config.rkt} + +to see, without building anything, the effect of the configuration in +@filepath{my-site-config.rkt} and the planned build steps. + +The default @exec{CONFIG} path is @filepath{build/site.rkt}, so you +could put your configuration file there and omit the @exec{CONFIG} +argument to @exec{make}. A default configuration file is created there +automatically. Supply @exec{CONFIG_MODE=...} to pass a configuration +mode on to your site-configuration module (accessible via the +@exec{current-mode} parameter). Supply @exec{CLEAN_MODE=--clean} to +make the default @racket[#:clean?] configuration for a client to +@racket[#t] instead of @racket[#f], supply +@exec{RELEASE_MODE=--release} to make the default @racket[#:release?] +configuration @racket[#t], supply @exec{SOURCE_MODE=--source} to make the +default @racket[#:source?] configuration @racket[#t], and supply +@exec{VERSIONLESS_MODE=--version} to make the default +@racket[#:versionless?] configuration @racket[#t]. + +A configuration file can specify the packages to include, host address +of the server, distribution name, installer directory, and +documentation search URL, but defaults can be provided as @exec{make} +arguments via @exec{PKGS}, @exec{SERVER} plus @exec{SERVER_PORT} plus +@exec{SERVER_HOSTS}, @exec{DIST_NAME}, @exec{DIST_BASE}, and +@exec{DIST_DIR}, @exec{DOC_SEARCH}, respectively. The site +configuration's top-level options for packages and documentation +search URL are used to configure the set of packages that are +available to client machines to include in installers. + +For each installer written to @filepath{build/installers}, the +installer's name is + +@centerline{@filepath{@nonterm{dist-base}-@nonterm{version}-@nonterm{platform}-@nonterm{dist-suffix}.@nonterm{ext}}} + +where @nonterm{dist-base} defaults to @filepath{racket} (but can be +set via @exec{DIST_BASE}), @nonterm{platform} is from +@racket[(system-library-subpath #f)] but normalizing the Windows +results to @filepath{i386-win32} and @filepath{x86_63-win32}, +-@nonterm{dist-suffix} is omitted unless a @racket[#:dist-suffix] +string is specified for the client in the site configuration, and +@nonterm{ext} is platform-specific: @filepath{.sh} for Unix (including +Linux), @filepath{.dmg} or @filepath{.pkg} for Mac OS, and +@filepath{.exe} for Windows. + +The server supports both @racket['cs] and @racket['3m] clients by +creating built packages in machine-independent form (which is then +recompiled to the client's native format, still much faster than +compiling from source). Set @exec{SERVER_COMPILE_MACHINE=} to disable +machine-independent format for built packages. + +@; ------------------------------------------------------------ +@section{Generating Installer Web Sites} + +The @exec{site} target of the makefile uses the @exec{installers} target to +generate a set of installers, and then it combines the installers, +packages, a package catalog, and log files into a directory that is +suitable for access via a web server. + +Supply the same @exec{CONFIG=...} and @exec{CONFIG_MODE=...} arguments for +@exec{site} as for @exec{installers}. The configuration file should have a +@racket[#:dist-base-url] entry for the URL where installers and packages will +be made available; the @exec{installers} target uses @racket[#:dist-base-url] to +embed suitable configuration into the installers. Specifically, +installers are configured to access pre-built packages and +documentation from the site indicated by @racket[#:dist-base-url]. + +Note that @racket[#:dist-base-url] should almost always end with +@filepath{/}, since others URLs will be constructed as relative to +@racket[#:dist-base-url]. + +The site is generated as @filepath{build/site} by default. A +@racket[#:site-dest] entry in the configuration file can select an +alternate destination. + +Use the @exec{site-from-installers} makefile target to perform the +part of @exec{site} that happens after @exec{installers} (i.e., to +generate a @exec{site} from an already-generated set of installers). + +@; ------------------------------------------------------------ +@section{Managing Snapshot Web Sites} + +The @exec{snapshot-site} makefile target uses @exec{site} (so supply +the same @exec{CONFIG=...} and @exec{CONFIG_MODE=...} arguments), and +then treats the resulting site as a snapshot with additional +snapshot-management tasks. + +For snapshot management, the destination of the files generated for +@exec{site} (as specified by @racket[#:site-dest]) should be within a +directory of snapshots. The configuration file can use +@racket[(current-stamp)] to get a string that represents the current +build, and then use the string both for @racket[#:dist-base-url] and +@racket[#:site-dest]. Normally, the stamp string is a combination of +the date and Git commit hash. + +Snapshot management includes creating an @filepath{index.html} file in +the snapshots directory (essentially a copy of the snapshot's own +@filepath{index.html}) and pruning snapshot subdirectories to keep the +number of snapshots at the amount specified by +@racket[#:max-snapshots] configuration-file entry (with a default +value of @racket[5]). + +Use the @exec{snapshot-at-site} makefile target to perform the part of +@exec{snapshot-site} that happens after @exec{site} (i.e., to manage +snapshots around an already-generated site). + +@; ------------------------------------------------------------ +@section{Separate Server and Clients} + +Instead of using the @exec{installers} makefile target and a site +configuration file, you can run server and client processes manually. + +Roughly, the steps are as follows + +@itemlist[#:style 'ordered + + @item{On the server machine: + + @commandline{make server PKGS="@nonterm{pkgs}"} + + See step 2 in the detailed steps below for more information on + variables other than @exec{PKGS} that you can provide with + @exec{make}.} + + @item{On each client machine: + + @commandline{make client SERVER=@nonterm{address} PKGS="@nonterm{pkgs}"} + + or + + @commandline{nmake win32-client SERVER=@nonterm{address} PKGS="@nonterm{pkgs}"} + + See 4 in the detailed steps below for more information on + variables other than @exec{SERVER} and @exec{PKGS} that you can + provide with @exec{make}.} + +] + +In more detail, the steps are as follows: + +@itemlist[#:style 'ordered + + + @item{Build @exec{racket} on a server. + + The @exec{base} target of the makefile will do that, if you + haven't done it already. (The server only works on non-Windows + platforms, currently.)} + + @item{On the server, build packages and start a catalog server. + + The @exec{server-from-base} target of the makefile will do that. + + Alternatively, use the @exec{server} target, which combines + @exec{base} and @exec{server-from-base} (i.e., steps 1 and 2). + + The @exec{SERVER_PORT} variable of the makefile choose the port + on which the server listens to clients. The default is port + @tt{9440}. + + The @exec{SERVER_HOSTS} variable of the makefile determines the + interfaces at which the server listens. The default is + @tt{localhost} which listens only on the loopback device (for + security). Supply the empty string to listen on all interfaces. + Supply multiple addresses by separating them with a comma. + + The @exec{PKGS} variable of the makefile determines which + packages are built for potential inclusion in a distribution. + + The @exec{DOC_SEARCH} variable of the makefile determine a URL + that is embedded in rendered documentation for cases where a + remote search is needed (because other documentation is not + installed). + + The @exec{SRC_CATALOG} variable determines the catalog that is + used to get package sources and native-library packages. The + default is @tt{http://pkgs.racket-lang.org}. + + The @exec{SERVER_PKG_INSTALL_OPTIONS} variable determines extra + flags that are passed to @exec{raco pkg install} when installing + on the server (to create package builds that are sent to + clients). For example, + @exec{SERVER_PKG_INSTALL_OPTIONS=--source} could be useful to + ensure that the server always builds from sources. + + The @exec{PACK_BUILT_OPTIONS} variable can be set to + @exec{--mode @nonterm{mode}} to set the package mode for built + packages. The default @exec{infer} mode infers uses the + package's @exec{distribution-preference} @filepath{info.rkt} + field, if any, infers @exec{binary} if the package has any + native libraries and no Racket sources, and infers @exec{built} + otherwise. + + The server provides README files from the + @filepath{build/readmes} directory. If @filepath{README.txt} + does not exist when the sever is started, a default file is + created (and clients download @filepath{README.txt} by default). + + If you stop the server and want to restart it, use the + @exec{built-package-server} makefile target instead of starting + over with the @exec{server} target.} + + @item{On each client (one for each platform to bundle), build @exec{racket}. + + This is the same as step 1, but on each client. If the client and + server are the same, there's nothing more to do for step 3.} + + @item{On each client, create an installer. + + The @exec{client} (or @exec{win32-client}) target of the + makefile will do that. + + Provide @exec{SERVER} as the hostname of the server machine, but + a @tt{localhost}-based tunnel back to the server is more secure + and avoids the need to specify @exec{SERVER_HOSTS} when starting + the server in step 2. Also, provide @exec{SERVER_PORT} if an + alternate port was specified in step 2. + + Provide the same @exec{PKGS} (or a subset) as in step 2 if you + want a different set than the ones listed in the makefile. + Similarly, @exec{DOC_SEARCH} normally should be the same as in + step 2, but for a client, it affects future documentation builds + in the installation. + + Alternatively, use the @exec{client} target, which combines + @exec{base} and @exec{client-from-base} (i.e., steps 3 and 4). + + On Windows, you need NSIS installed, either in the usual location + or with @exec{makensis} in your command-line path. + + To create a release installer, provide @exec{RELEASE_MODE} as + @DFlag{release} to @exec{make}. A release installer has slightly + different defaults that are suitable for infrequently updated + release installations, as opposed to frequently updated snapshot + installations. + + To create a source archive, provide @exec{SOURCE_MODE} as + @DFlag{source} to @exec{make}. + + To create an archive that omits the version number and also omit + and version number in installer paths, provide + @exec{VERSIONLESS_MODE} as @DFlag{versionless} to @exec{make}. + + To change the human-readable name of the distribution as + embedded in the installer, provide @exec{DIST_NAME} to + @exec{make}. The default distribution name is @tt{Racket}. + Whatever name you pick, the Racket version number is + automatically added for various contexts. + + To change the base name of the installer file, provide + @exec{DIST_BASE} to @exec{make}. The default is @tt{racket}. + + To change the directory name for installation on Unix (including + Linux), provide @exec{DIST_DIR} to @exec{make}. The default is + @tt{racket}. + + To add an extra piece to the installer's name, such as an + identifier for a variant of Linux, provide @exec{DIST_SUFFIX} to + @exec{make}. The default is @tt{"", which} omits the prefix and + its preceding hyphen. + + To set the description string for the installer, provide + @exec{DIST_DESC} to @exec{make}. The description string is + recorded alongside the installer. + + To set the initial package catalogs URLs for an installation, + provide @exec{DIST_CATALOGS_q} to @exec{make}. Separate multiple + URLs with a space, and use an empty string in place of a URL to + indicate that the default catalogs should be used. The @tt{_q} + in the variable name indicates that its value can include double + quotes (but not single quotes)---which are needed to specify an + empty string, for example. + + To select a @filepath{README} file for the client, provide + @exec{README} to @exec{make}. The @exec{README} value is used as + a file name to download from the server. + + To create a @filepath{.tgz} archive instead of an installer (or + any platform), set @exec{TGZ_MODE} to @DFlag{tgz}. + + For a Mac OS installer, set @exec{SIGN_IDENTITY} as the name to + which the signing certificate is associated. Set + @exec{MAC_PKG_MODE} to @DFlag{mac-pkg} to create a + @filepath{.pkg} installer instead of a @filepath{.dmg} image. + + For a Windows installer, set @exec{OSSLSIGNCODE_ARGS_BASE64} as + a Base64 encoding of an S-expression for a list of argument + strings for @exec{osslsigncode}. The @Flag{n}, @Flag{t}, + @Flag{in}, and @Flag{out} arguments are provided to + @exec{osslsigncode} automatically, so supply the others. + + The @exec{SERVER_CATALOG_PATH} and @exec{SERVER_COLLECTS_PATH} + makefile variables specify paths at @exec{SERVER} plus + @exec{SERVER_PORT} to access the package catalog and pre-built + @filepath{collects} tree needed for a client, but those paths + should be empty for a server started with @exec{make server}, + and they are used mainly by @exec{make client-from-site} + (described below). + + The @exec{UPLOAD} makefile variable specifies a URL to use as an + upload destination for the created installed, where the + installer's name is added to the end of the URL, or leave as + empty for no upload.} + +] + +On each client, step 4 produces a @filepath{bundle/installer.txt} file +that contains the path to the generated installer on one line, +followed by the description on a second line. The installer is also +uploaded to the server, which leaves the installer in a +@filepath{build/installers} directory and records a mapping from the +installer's description to its filename in +@filepath{build/installers/table.rktd}. + +If you provide @exec{JOB_OPTIONS=@nonterm{options}} for either a +client or server build, the options are used both for @exec{raco +setup} and @exec{raco pkg install}. Normally, @exec{JOB_OPTIONS} is +used to control parallelism. + +@; ------------------------------------------------------------ +@section{Creating a Client from an Installer Web Site} + +If you (or someone else) previously created an installer site with +@exec{make site}, then @exec{make client-from-site} in a clean +repository creates an installer for the current platform drawing +packages from the site. + +At a minimum, provide @exec{SERVER}, @exec{SERVER_PORT} (usually 80), +and @exec{SITE_PATH} (if not empty, include a trailing @litchar{/}) +makefile variables to access a site at + +@centerline{@tt{http://$(SERVER):$(SERVER_PORT)/$(SITE_PATH)}} + +The @exec{client-from-site} makefile target chains to @exec{make +client} while passing suitable values for @exec{DIST_CATALOGS_q}, +@exec{DOC_SEARCH}, @exec{SERVER_CATALOG_PATH}, and +@exec{SERVER_COLLECTS_PATH}. Supply any other suitable variables, such +as @exec{DIST_NAME} or @exec{RELEASE_MODE}, the same as for @exec{make +client}. diff --git a/pkgs/racket-build-guide/info.rkt b/pkgs/racket-build-guide/info.rkt new file mode 100644 index 0000000000..2e7b575505 --- /dev/null +++ b/pkgs/racket-build-guide/info.rkt @@ -0,0 +1,16 @@ +#lang info + +(define scribblings '(("racket-build-guide.scrbl" (multi-page) (racket-core -98)))) + +(define deps + '("base")) + +(define build-deps + '("scribble-lib" + "racket-doc" + "scribble-doc" + "distro-build-doc")) + +(define pkg-desc "Racket build and contribution documentation") + +(define pkg-authors '(mflatt)) diff --git a/pkgs/racket-build-guide/racket-build-guide.scrbl b/pkgs/racket-build-guide/racket-build-guide.scrbl new file mode 100644 index 0000000000..d9e2425b9d --- /dev/null +++ b/pkgs/racket-build-guide/racket-build-guide.scrbl @@ -0,0 +1,18 @@ +#lang scribble/manual +@(require "common.rkt") + +@title{Building, Distributing, and Contributing to Racket} + +The main Racket source code repository is + +@centerline{@url[git-repo]} @; `git-repo` is defined in "common.rkt" + +This guide explains how to build those sources, how to create Racket +distributions like the ones at @url{https://download.racket-lang.org}, and +how to contribute to Racket development. + +@table-of-contents[] + +@include-section["build.scrbl"] +@include-section["distribute.scrbl"] +@include-section["contribute.scrbl"]