note about perl dependency

LICENSE.md

@@ -0,0 +1,2 @@
+* TeX Live: <https://www.tug.org/texlive/copying.html>
+* LaTeX: <https://www.latex-project.org/lppl.txt>

Makefile

@@ -28,7 +28,16 @@ example.pdf:
 
 STACK_NAME ?= latex-layer 
 
-build/output.yaml: template.yaml result/bin/x86_64-linux/latex
+#result/texlive/bin/x86_64-linux/pdflatex: all
+
+build/layer.zip: result/texlive/bin/x86_64-linux/pdflatex build
+	# CloudFormation has trouble zipping texlive due to nested directory dept
+	#
+	# This is why we zip outside
+	
+	cd result && zip -ry $(PROJECT_ROOT)$@ *
+
+build/output.yaml: template.yaml build/layer.zip
 	aws cloudformation package --template $< --s3-bucket $(DEPLOYMENT_BUCKET) --output-template-file $@
 
 deploy: build/output.yaml

Makefile_Latex

@@ -9,7 +9,7 @@ $(TEXLIVE_SOURCE):
 	curl -LO http://mirror.ctan.org/systems/texlive/tlnet/$(TEXLIVE_SOURCE)
 
 $(MD5_pm):
-	yum install perl-Digest-MD5 -y
+	yum install -y perl-Digest-MD5
 
 $(WGET):
 	yum install -y wget

README-SAR.md

@@ -0,0 +1,20 @@
+# TeX Live (latex/pdflatex) for AWS Lambda
+
+TeX Live (including `latex` and `pdflatex`) for AWS Lambda, including packages and fonts required for Pandoc.
+
+Intended for instances powered by Amazon Linux 2.x, such as the `nodejs10.x` runtime, and the updated 2018.03 Amazon Linux 1 runtimes.
+
+The binaries will be in `/opt/texlive/bin/x86_64-linux` after linking the layer to a Lambda function. *Binaries depend on each other, so when executing, you will need to add that directory to your `PATH` environment variable*.
+
+```
+pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019)
+kpathsea version 6.3.1
+Compiled with libpng 1.6.36; using libpng 1.6.36
+Compiled with zlib 1.2.11; using zlib 1.2.11
+Compiled with xpdf version 4.01
+```
+
+Note that this distribution comes with minimal fonts ([lm](https://ctan.org/pkg/lm?lang=en), [amsfonts](https://ctan.org/pkg/amsfonts?lang=en) and [ec](https://ctan.org/pkg/ec?lang=en)), required for Pandoc. To add more fonts, modify the build profile from <https://github.com/serverlesspub/latex-aws-lambda-layer/>.
+
+For an example of how to use the layer, check out 
+<https://github.com/serverlesspub/latex-aws-lambda-layer/example>.

README.md

@@ -0,0 +1,95 @@
+# TeX Live (LaTeX/pdflatex) for AWS Lambda
+
+TeX Live (including `latex` and `pdflatex`) for AWS Lambda, including packages and fonts required for Pandoc.
+
+Intended for instances powered by Amazon Linux 2.x, such as the `nodejs10.x` runtime, and the updated 2018.03 Amazon Linux 1 runtimes
+
+## WORK IN PROGRESS!!!
+
+Note that this is work in progress, not ready for public usage yet. TeX Live requires a functioning `perl` executable, even at runtime, and this layer does not include that. 
+
+## Usage
+
+After deployment, the binaries will be in `/opt/texlive/bin/x86_64-linux` after linking the layer to a Lambda function. 
+
+Binaries depend on each other, so when executing, you will need to add that directory to your `PATH` environment variable.
+
+## Prerequisites
+
+* Docker desktop
+* Unix Make environment
+* AWS command line utilities (just for deployment)
+
+## Compiling the code
+
+* start Docker services
+* `make all`
+
+There are two `make` scripts in this project.
+
+* [`Makefile`](Makefile) is intended to run on the build system, and just starts a Docker container matching the AWS Linux 2 environment for Lambda runtimes to compile Latex using the second script.
+* [`Makefile_Latex`](Makefile_Latex) is the script that will run inside the container, and actually compile binaries. 
+
+The output will be in the `result` dir.
+
+### Configuring the build
+
+By default, this compiles a version expecting to run as a Lambda layer from `/opt/texlive`. Change the expected runtime location in [`texlive.profile`](texlive.profile).
+
+The default Docker image used is `lambci/lambda-base-2:build`. To use a different base, provide a `DOCKER_IMAGE` variable when invoking `make`.
+
+You can include/exclude collections from [`texlive.profile`](texlive.profile), or change the CTAN packages installed in addition to the minimal collection in [`Makefile_Latex`](Makefile_Latex).
+
+Note that this distribution comes with minimal fonts ([lm](https://ctan.org/pkg/lm?lang=en), [amsfonts](https://ctan.org/pkg/amsfonts?lang=en) and [ec](https://ctan.org/pkg/ec?lang=en)) required for Pandoc. To add the full recommended font set, enable `collection-fontsrecommended` in [`texlive.profile`](texlive.profile).
+
+### Experimenting
+
+* `make bash` to open an interactive shell with all the build directories mounted
+
+### Compiled info
+
+```
+pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019)
+kpathsea version 6.3.1
+Compiled with libpng 1.6.36; using libpng 1.6.36
+Compiled with zlib 1.2.11; using zlib 1.2.11
+Compiled with xpdf version 4.01
+```
+
+## Deploying to AWS as a layer
+
+Run the following command to deploy the compiled result as a layer in your AWS account.
+
+```
+make deploy DEPLOYMENT_BUCKET=<YOUR BUCKET NAME>
+```
+
+### configuring the deployment
+
+By default, this uses `latex-layer` as the stack name. Provide a `STACK_NAME` variable when
+calling `make deploy` to use an alternative name.
+
+### example usage
+
+An example project is in the [example](example) directory. It sets up two buckets, and listens to file uploads on the first bucket to convert and generate HTML files from markdown. You can deploy it from the root Makefile using:
+
+```
+make deploy-example DEPLOYMENT_BUCKET=<YOUR BUCKET NAME>
+```
+
+For more information, check out:
+
+* general install process: <https://www.tug.org/texlive/quickinstall.html>
+* profile variables: <https://wiki.archlinux.org/index.php/TeX_Live>
+* required packages for pandoc: <https://pandoc.org/MANUAL.html>
+
+## Author
+
+Gojko Adzic <https://gojko.net>
+
+## License
+
+* These scripts: [MIT](https://opensource.org/licenses/MIT)
+* TeX Live: <https://www.tug.org/texlive/copying.html>
+* LaTeX: <https://www.latex-project.org/lppl.txt>
+* Contained libraries all have separate licenses, check the respective web sites for more information

example/.gitignore

@@ -0,0 +1 @@
+output.yaml

example/Makefile

@@ -0,0 +1,17 @@
+STACK_NAME ?= latex-layer-example
+LATEX_STACK_NAME ?= latex-layer
+
+LATEX_LAYER ?=$(shell aws cloudformation describe-stacks --stack-name $(LATEX_STACK_NAME) --query Stacks[].Outputs[].OutputValue --output text)
+SOURCES=$(shell find src/)
+
+clean: 
+	rm -rf build
+
+output.yaml: template.yaml $(SOURCES)
+	mkdir -p build
+	aws cloudformation package --template-file $< --output-template-file $@ --s3-bucket $(DEPLOYMENT_BUCKET)
+
+deploy: output.yaml
+	aws cloudformation deploy --template-file $< --stack-name $(STACK_NAME) --capabilities CAPABILITY_IAM --parameter-overrides LatexLayer=$(LATEX_LAYER)
+	aws cloudformation describe-stacks --stack-name $(STACK_NAME) --query Stacks[].Outputs --output table
+

example/src/child-process-promise.js

@@ -0,0 +1,26 @@
+/*global module, require, console, Promise */
+'use strict';
+const childProcess = require('child_process'),
+	spawnPromise = function (command, argsarray, envOptions) {
+		return new Promise((resolve, reject) => {
+			console.log('executing', command, argsarray.join(' '));
+			const childProc = childProcess.spawn(command, argsarray, envOptions || {env: process.env, cwd: process.cwd()}),
+				resultBuffers = [];
+			childProc.stdout.on('data', buffer => {
+				console.log(buffer.toString());
+				resultBuffers.push(buffer);
+			});
+			childProc.stderr.on('data', buffer => console.error(buffer.toString()));
+			childProc.on('exit', (code, signal) => {
+				console.log(`${command} completed with ${code}:${signal}`);
+				if (code || signal) {
+					reject(`${command} failed with ${code || signal}`);
+				} else {
+					resolve(Buffer.concat(resultBuffers).toString().trim());
+				}
+			});
+		});
+	};
+module.exports = {
+	spawn: spawnPromise
+};

example/src/index.js

@@ -0,0 +1,28 @@
+
+const s3Util = require('./s3-util'),
+	childProcessPromise = require('./child-process-promise'),
+	path = require('path'),
+	os = require('os'),
+	OUTPUT_BUCKET = process.env.OUTPUT_BUCKET;
+
+exports.handler = function (eventObject, context) {
+	const eventRecord = eventObject.Records && eventObject.Records[0],
+		inputBucket = eventRecord.s3.bucket.name,
+		key = eventRecord.s3.object.key,
+		id = context.awsRequestId,
+		extension = '.pdf',
+		resultKey = key.replace(/\.[^.]+$/, extension),
+		workdir = os.tmpdir(),
+		inputFile = path.join(workdir,  id + path.extname(key)),
+		outputFile = path.join(workdir, id + extension);
+
+
+	console.log('converting', inputBucket, key, 'using', inputFile);
+	return s3Util.downloadFileFromS3(inputBucket, key, inputFile)
+		.then(() => childProcessPromise.spawn(
+			'pdflatex',
+			[inputFile],
+			{env: process.env, cwd: workdir}
+		))
+		.then(() => s3Util.uploadFileToS3(OUTPUT_BUCKET, resultKey, outputFile, MIME_TYPE));
+};

example/src/s3-util.js

@@ -0,0 +1,38 @@
+/*global module, require, Promise, console */
+
+const aws = require('aws-sdk'),
+	fs = require('fs'),
+	s3 = new aws.S3(),
+	downloadFileFromS3 = function (bucket, fileKey, filePath) {
+		'use strict';
+		console.log('downloading', bucket, fileKey, filePath);
+		return new Promise(function (resolve, reject) {
+			const file = fs.createWriteStream(filePath),
+				stream = s3.getObject({
+					Bucket: bucket,
+					Key: fileKey
+				}).createReadStream();
+			stream.on('error', reject);
+			file.on('error', reject);
+			file.on('finish', function () {
+				console.log('downloaded', bucket, fileKey);
+				resolve(filePath);
+			});
+			stream.pipe(file);
+		});
+	}, uploadFileToS3 = function (bucket, fileKey, filePath, contentType) {
+		'use strict';
+		console.log('uploading', bucket, fileKey, filePath);
+		return s3.upload({
+			Bucket: bucket,
+			Key: fileKey,
+			Body: fs.createReadStream(filePath),
+			ACL: 'private',
+			ContentType: contentType
+		}).promise();
+	};
+
+module.exports = {
+	downloadFileFromS3: downloadFileFromS3,
+	uploadFileToS3: uploadFileToS3
+};

example/template.yaml

@@ -0,0 +1,47 @@
+AWSTemplateFormatVersion: 2010-09-09
+Transform: AWS::Serverless-2016-10-31
+Description: >
+  Example project demonstrating the usage of the Latex Layer for AWS Linux 2 runtimes.
+
+Parameters:
+  LatexLayer:
+    Type: String
+Resources:
+  UploadBucket:
+    Type: AWS::S3::Bucket
+
+  ResultsBucket:
+    Type: AWS::S3::Bucket
+
+  ConvertFileFunction:
+    Type: AWS::Serverless::Function
+    Properties:
+      Handler: index.handler
+      Timeout: 180
+      MemorySize: 1024
+      Runtime: nodejs10.x
+      CodeUri: src
+      Layers:
+        - !Ref LatexLayer
+      Policies:
+      - S3CrudPolicy:
+          BucketName: !Sub "${AWS::StackName}-*"
+      Environment:
+        Variables:
+          OUTPUT_BUCKET: !Ref ResultsBucket
+          PATH: '/opt/texlive/bin/x86_64-linux:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin'
+      Events:
+        FileUpload:
+          Type: S3
+          Properties:
+            Bucket: !Ref UploadBucket
+            Events: s3:ObjectCreated:*
+
+Outputs:
+    UploadBucket:
+      Description: "Upload S3 bucket"
+      Value: !Ref UploadBucket
+    ResultsBucket:
+      Description: "Results S3 bucket"
+      Value: !Ref ResultsBucket
+

template.yaml

@@ -0,0 +1,40 @@
+AWSTemplateFormatVersion: 2010-09-09
+Transform: AWS::Serverless-2016-10-31
+Description: >
+  TeXLive (LaTeX, pdf-latex) for Amazon Linux 2,
+  including packages and fonts required for Pandoc.
+
+  Check out https://github.com/serverlesspub/latex-aws-lambda-layer
+  for more information.
+Resources:
+  LatexLayer:
+    Type: AWS::Serverless::LayerVersion
+    Properties:
+      LayerName: latex
+      Description: 'TeXLive (LaTeX, pdf-latex) for Amazon Linux 2'
+      ContentUri: build/layer.zip
+      CompatibleRuntimes:
+        - nodejs10.x
+      LicenseInfo: https://www.tug.org/texlive/LICENSE.TL
+      RetentionPolicy: Retain
+
+Outputs:
+  LayerVersion:
+    Description: Layer ARN Reference
+    Value: !Ref LatexLayer
+
+Metadata:
+  AWS::ServerlessRepo::Application:
+    Name: latex-lambda-layer
+    Description: >
+      TeXLive (LaTeX, pdf-latex) for Amazon Linux 2,
+      including packages and fonts required for Pandoc.
+
+    Author: Gojko Adzic
+    SpdxLicenseId: Latex2e
+    LicenseUrl: LICENSE.md
+    ReadmeUrl: README-SAR.md 
+    Labels: ['layer', 'latex', 'pdflatex', 'lambda', 'texlive', 'pandoc']
+    HomePageUrl: https://github.com/serverlesspub/latex-aws-lambda-layer
+    SemanticVersion: 1.0.0
+    SourceCodeUrl: https://github.com/serverlesspub/latex-aws-lambda-layer