Summary:
This diff provides support for Latin-1, Cyrillic, and CJK characters
inside \text{} groups. For Latin-1 and Cyrillic characters we use
glyph metrics from a glyph from Basic Latin that has roughly the same
bounding box. We use the metrics for a capital 'M' to approximate the
full-width CJK characters. Half-width characters are not supported yet.
Test Plan:
- make test
- make screenshots
Reviewers: emily
This adds support for the following input sequences:
-- --- ` ' `` '' \degree \pounds \maltese
resulting in – — ‘ ’ “ ” ° £ ✠ symbols already present in our fonts.
As part of this modification, the recognition of multiple dashes was moved
from the lexer to the parser.
This is neccessary since in math mode a sequence of hyphens is just a
sequence of minus signs. Just like a pair of apostrophes in math mode is a
double prime not a right double quotation mark.
To make this easier, parseGroup and parseOptionalGroup have been merged.
* Introduce MacroExpander
The job of the MacroExpander is turning a stream of possibly expandable
tokens, as obtained from the Lexer, into a stream of non-expandable tokens
(in KaTeX, even though they may well be expandable in TeX) which can be
processed by the Parser. The challenge here is that we don't have
mode-specific lexer implementations any more, so we need to do everything on
the token level, including reassembly of sizes and colors.
* Make macros available in development server
Now one can specify macro definitions like \foo=bar as part of the query
string and use these macros in the formula being typeset.
* Add tests for macro expansions
* Handle end of input in special groups
This avoids an infinite loop if input ends prematurely.
* Simplify parseSpecialGroup
The parseSpecialGroup methos now returns a single token spanning the whole
special group, and leaves matching that string against a suitable regular
expression to whoever is calling the method. Suggested by @cbreeden.
* Incorporate review suggestions
Add improvements suggested by Kevin Barabash during review.
* Input range sanity checks
Ensure that both tokens of a token range come from the same lexer,
and that the range has a non-negative length.
* Improved wording of two comments
Summary
We'd like contributors to use the same linter and lint rules that we use
internally. This diff swaps out eslint for jshint and fixes all lint failures
except for the max-len failures in the test suites.
Test Plan:
- ka-lint src
- make lint
- make test
Reviewers: emily
Instead of passing around the current position as an argument, we now have a
parser property called pos to keep track of that. Instead of repeatedly
re-lexing at the current position we now have a property called nextToken
which contains the token beginning at the current position. We may need to
re-lex if we switch mode. Since the position is kept in the parser state,
we don't need to return it from parsing methods, which obsoletes the
ParseResult class.
Fixes issue #255.
Mixing the variable number of arguments a function receives from TeX code
with the fixed arguments which the parser provides can cause some confusion.
After this change, a handler will receive exactly two arguments: one is a
context object from which things provided by the parser can be accessed by
name, which allows for simple extensions in the future. The other is the
list of TeX arguments, passed as an array.
If we ever switch to EcmaScript 2015, we might want to use its destructuring
features to name the elements of the args array in the function head. Until
then, destructuring that array manually immediately at the beginning of the
function seems like a useful convention to easily find the meaning of these
arguments.
Also, the MathBb-chrome test changed, to what I believe is the correct
result? Not sure why it looked wrong before.
Test plan:
- `make test`
- take screenshots, see nothing changed.
This commit introduces environments, and implements the parser
infrastructure to handle them, even including arguments after the
“\begin{name}” construct. It also offers a way to turn array-like data
structures, i.e. delimited by “&” and “\\”, into nested arrays of groups.
Environments are essentially functions which call back to the parser to
parse their body. It is their responsibility to stop at the next “\end”,
while the parser takes care of verifing that the names match between
“\begin” and “\end”. The environment has to return a ParseResult, to
provide the position that goes with the resulting node.
One application of this is the “array” environment. So far, it supports
column alignment, but no column separators, and no multi-column shorthands
using “*{…}”. Building on the same infrastructure, there are “matrix”,
“pmatrix”, “bmatrix”, “vmatrix” and “Vmatrix” environments. Internally
these are just “\left..\right” wrapped around an array with no margins at
its ends. Spacing for arrays and matrices was derived from the LaTeX
sources, and comments indicate the appropriate references.
Now we have hard-wired breaks in parseExpression, to always break on “}”,
“\end”, “\right”, “&”, “\\” and “\cr”. This means that these symbols are
never PART of an expression, at least not without some nesting. They may
follow AFTER an expression, and the caller of parseExpression should be
expecting them. The implicit groups for sizing or styling don't care what
ended the expression, which is all right for them. We still have support
for breakOnToken, but now it is only used for “]” since that MAY be used to
terminate an optional argument, but otherwise it's an ordinary symbol.
Summary:
Add the ability to pass in options to the render calls which contain information about the parse. This information is passed around to the parser and builder, which parse and render differently depending on the options. Currently, this includes an option to render the math in display mode (i.e. centered, block level, and in displaystyle).
Also added some changes to make it easier to add new data to functions (now that new data doesn't need to be copied into the ParseFuncOrArg data structure, it is looked up when it is needed) and has more sane support for the `'original'` argType (as suggested by pull request #93).
Test Plan:
- Make sure tests and lint pass
- Make sure huxley screenshots didn't change, and new screenshot looks correct
Reviewers: alpert
Reviewed By: alpert
Differential Revision: https://phabricator.khanacademy.org/D13810
Summary:
Add correct parsing of optional arguments. Now, things like `\rule` can shift
based on its argument, and parsing of `\sqrt[3]{x}` fails (correctly) because we
don't support that yet.
Also, cleaned up the lexing code a bit. There was a vestige of the old types in
the lexer (they have now been completely moved to symbols.js). As a byproduct,
this made it hard to call `expect("]")`, because it would look at the type of
the Token and the type for "]" was "close". Now, all functions just look at the
text of the parsed token, and in special occasions (like in the dimension lexer)
it can return some data along with it.
Test Plan:
- Make sure tests still work, and new tests work
- Make sure no huxley screenshots changed
- Make EXTRA SURE `\sqrt[3]{x}` fails.
Reviewers: alpert
Reviewed By: alpert
Differential Revision: http://phabricator.khanacademy.org/D13505
changed stopType (string) parameter to breakOnInfix (boolean)
renamed rewriteInfixNodes to handleInfixNodes
added a test for {1 \over 2} \over 3, fixed some grammar, and added code in the parser to squash superfluous ordgroups
removed squashOrdGroups and instead don't create an "ordgroup" if one already exists
removed unnecessary variable
moved variable declarations out of "if" statements
removed comment
Fixed style issue with where variables are declared and remove unnecessary comment from functions.js
