239 lines
11 KiB
HTML
239 lines
11 KiB
HTML
|
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<HTML><HEAD><TITLE>Man page of PCRE2SERIALIZE</TITLE>
|
|
</HEAD><BODY>
|
|
<H1>PCRE2SERIALIZE</H1>
|
|
Section: C Library Functions (3)<BR>Updated: 27 June 2018<BR><A HREF="#index">Index</A>
|
|
<A HREF="/cgi-bin/man/man2html">Return to Main Contents</A><HR>
|
|
|
|
<A NAME="lbAB"> </A>
|
|
<H2>NAME</H2>
|
|
|
|
PCRE2 - Perl-compatible regular expressions (revised API)
|
|
<A NAME="lbAC"> </A>
|
|
<H2>SAVING AND RE-USING PRECOMPILED PCRE2 PATTERNS</H2>
|
|
|
|
|
|
<P>
|
|
<PRE>
|
|
<B>int32_t pcre2_serialize_decode(pcre2_code **</B><I>codes</I>,
|
|
<B> int32_t </B><I>number_of_codes</I>, const uint32_t *<I>bytes</I>,
|
|
<B> pcre2_general_context *</B><I>gcontext</I>);
|
|
|
|
<B>int32_t pcre2_serialize_encode(pcre2_code **</B><I>codes</I>,
|
|
<B> int32_t </B><I>number_of_codes</I>, uint32_t **<I>serialized_bytes</I>,
|
|
<B> PCRE2_SIZE *</B><I>serialized_size</I>, pcre2_general_context *<I>gcontext</I>);
|
|
|
|
<B>void pcre2_serialize_free(uint8_t *</B><I>bytes</I>);
|
|
|
|
<B>int32_t pcre2_serialize_get_number_of_codes(const uint8_t *</B><I>bytes</I>);
|
|
</PRE>
|
|
|
|
<P>
|
|
If you are running an application that uses a large number of regular
|
|
expression patterns, it may be useful to store them in a precompiled form
|
|
instead of having to compile them every time the application is run. However,
|
|
if you are using the just-in-time optimization feature, it is not possible to
|
|
save and reload the JIT data, because it is position-dependent. The host on
|
|
which the patterns are reloaded must be running the same version of PCRE2, with
|
|
the same code unit width, and must also have the same endianness, pointer width
|
|
and PCRE2_SIZE type. For example, patterns compiled on a 32-bit system using
|
|
PCRE2's 16-bit library cannot be reloaded on a 64-bit system, nor can they be
|
|
reloaded using the 8-bit library.
|
|
<P>
|
|
|
|
Note that "serialization" in PCRE2 does not convert compiled patterns to an
|
|
abstract format like Java or .NET serialization. The serialized output is
|
|
really just a bytecode dump, which is why it can only be reloaded in the same
|
|
environment as the one that created it. Hence the restrictions mentioned above.
|
|
Applications that are not statically linked with a fixed version of PCRE2 must
|
|
be prepared to recompile patterns from their sources, in order to be immune to
|
|
PCRE2 upgrades.
|
|
<A NAME="lbAD"> </A>
|
|
<H2>SECURITY CONCERNS</H2>
|
|
|
|
|
|
<P>
|
|
The facility for saving and restoring compiled patterns is intended for use
|
|
within individual applications. As such, the data supplied to
|
|
<B>pcre2_serialize_decode()</B> is expected to be trusted data, not data from
|
|
arbitrary external sources. There is only some simple consistency checking, not
|
|
complete validation of what is being re-loaded. Corrupted data may cause
|
|
undefined results. For example, if the length field of a pattern in the
|
|
serialized data is corrupted, the deserializing code may read beyond the end of
|
|
the byte stream that is passed to it.
|
|
<A NAME="lbAE"> </A>
|
|
<H2>SAVING COMPILED PATTERNS</H2>
|
|
|
|
|
|
<P>
|
|
Before compiled patterns can be saved they must be serialized, which in PCRE2
|
|
means converting the pattern to a stream of bytes. A single byte stream may
|
|
contain any number of compiled patterns, but they must all use the same
|
|
character tables. A single copy of the tables is included in the byte stream
|
|
(its size is 1088 bytes). For more details of character tables, see the
|
|
|
|
|
|
section on locale support
|
|
|
|
in the
|
|
|
|
<B>pcre2api</B>
|
|
|
|
documentation.
|
|
<P>
|
|
|
|
The function <B>pcre2_serialize_encode()</B> creates a serialized byte stream
|
|
from a list of compiled patterns. Its first two arguments specify the list,
|
|
being a pointer to a vector of pointers to compiled patterns, and the length of
|
|
the vector. The third and fourth arguments point to variables which are set to
|
|
point to the created byte stream and its length, respectively. The final
|
|
argument is a pointer to a general context, which can be used to specify custom
|
|
memory mangagement functions. If this argument is NULL, <B>malloc()</B> is used
|
|
to obtain memory for the byte stream. The yield of the function is the number
|
|
of serialized patterns, or one of the following negative error codes:
|
|
<P>
|
|
<BR> PCRE2_ERROR_BADDATA the number of patterns is zero or less
|
|
<BR> PCRE2_ERROR_BADMAGIC mismatch of id bytes in one of the patterns
|
|
<BR> PCRE2_ERROR_MEMORY memory allocation failed
|
|
<BR> PCRE2_ERROR_MIXEDTABLES the patterns do not all use the same tables
|
|
<BR> PCRE2_ERROR_NULL the 1st, 3rd, or 4th argument is NULL
|
|
<P>
|
|
PCRE2_ERROR_BADMAGIC means either that a pattern's code has been corrupted, or
|
|
that a slot in the vector does not point to a compiled pattern.
|
|
<P>
|
|
|
|
Once a set of patterns has been serialized you can save the data in any
|
|
appropriate manner. Here is sample code that compiles two patterns and writes
|
|
them to a file. It assumes that the variable <I>fd</I> refers to a file that is
|
|
open for output. The error checking that should be present in a real
|
|
application has been omitted for simplicity.
|
|
<P>
|
|
<BR> int errorcode;
|
|
<BR> uint8_t *bytes;
|
|
<BR> PCRE2_SIZE erroroffset;
|
|
<BR> PCRE2_SIZE bytescount;
|
|
<BR> pcre2_code *list_of_codes[2];
|
|
<BR> list_of_codes[0] = pcre2_compile("first pattern",
|
|
<BR> PCRE2_ZERO_TERMINATED, 0, &errorcode, &erroroffset, NULL);
|
|
<BR> list_of_codes[1] = pcre2_compile("second pattern",
|
|
<BR> PCRE2_ZERO_TERMINATED, 0, &errorcode, &erroroffset, NULL);
|
|
<BR> errorcode = pcre2_serialize_encode(list_of_codes, 2, &bytes,
|
|
<BR> &bytescount, NULL);
|
|
<BR> errorcode = fwrite(bytes, 1, bytescount, fd);
|
|
<P>
|
|
Note that the serialized data is binary data that may contain any of the 256
|
|
possible byte values. On systems that make a distinction between binary and
|
|
non-binary data, be sure that the file is opened for binary output.
|
|
<P>
|
|
|
|
Serializing a set of patterns leaves the original data untouched, so they can
|
|
still be used for matching. Their memory must eventually be freed in the usual
|
|
way by calling <B>pcre2_code_free()</B>. When you have finished with the byte
|
|
stream, it too must be freed by calling <B>pcre2_serialize_free()</B>. If this
|
|
function is called with a NULL argument, it returns immediately without doing
|
|
anything.
|
|
<A NAME="lbAF"> </A>
|
|
<H2>RE-USING PRECOMPILED PATTERNS</H2>
|
|
|
|
|
|
<P>
|
|
In order to re-use a set of saved patterns you must first make the serialized
|
|
byte stream available in main memory (for example, by reading from a file). The
|
|
management of this memory block is up to the application. You can use the
|
|
<B>pcre2_serialize_get_number_of_codes()</B> function to find out how many
|
|
compiled patterns are in the serialized data without actually decoding the
|
|
patterns:
|
|
<P>
|
|
<BR> uint8_t *bytes = <serialized data>;
|
|
<BR> int32_t number_of_codes = pcre2_serialize_get_number_of_codes(bytes);
|
|
<P>
|
|
The <B>pcre2_serialize_decode()</B> function reads a byte stream and recreates
|
|
the compiled patterns in new memory blocks, setting pointers to them in a
|
|
vector. The first two arguments are a pointer to a suitable vector and its
|
|
length, and the third argument points to a byte stream. The final argument is a
|
|
pointer to a general context, which can be used to specify custom memory
|
|
mangagement functions for the decoded patterns. If this argument is NULL,
|
|
<B>malloc()</B> and <B>free()</B> are used. After deserialization, the byte
|
|
stream is no longer needed and can be discarded.
|
|
<P>
|
|
<BR> int32_t number_of_codes;
|
|
<BR> pcre2_code *list_of_codes[2];
|
|
<BR> uint8_t *bytes = <serialized data>;
|
|
<BR> int32_t number_of_codes =
|
|
<BR> pcre2_serialize_decode(list_of_codes, 2, bytes, NULL);
|
|
<P>
|
|
If the vector is not large enough for all the patterns in the byte stream, it
|
|
is filled with those that fit, and the remainder are ignored. The yield of the
|
|
function is the number of decoded patterns, or one of the following negative
|
|
error codes:
|
|
<P>
|
|
<BR> PCRE2_ERROR_BADDATA second argument is zero or less
|
|
<BR> PCRE2_ERROR_BADMAGIC mismatch of id bytes in the data
|
|
<BR> PCRE2_ERROR_BADMODE mismatch of code unit size or PCRE2 version
|
|
<BR> PCRE2_ERROR_BADSERIALIZEDDATA other sanity check failure
|
|
<BR> PCRE2_ERROR_MEMORY memory allocation failed
|
|
<BR> PCRE2_ERROR_NULL first or third argument is NULL
|
|
<P>
|
|
PCRE2_ERROR_BADMAGIC may mean that the data is corrupt, or that it was compiled
|
|
on a system with different endianness.
|
|
<P>
|
|
|
|
Decoded patterns can be used for matching in the usual way, and must be freed
|
|
by calling <B>pcre2_code_free()</B>. However, be aware that there is a potential
|
|
race issue if you are using multiple patterns that were decoded from a single
|
|
byte stream in a multithreaded application. A single copy of the character
|
|
tables is used by all the decoded patterns and a reference count is used to
|
|
arrange for its memory to be automatically freed when the last pattern is
|
|
freed, but there is no locking on this reference count. Therefore, if you want
|
|
to call <B>pcre2_code_free()</B> for these patterns in different threads, you
|
|
must arrange your own locking, and ensure that <B>pcre2_code_free()</B> cannot
|
|
be called by two threads at the same time.
|
|
<P>
|
|
|
|
If a pattern was processed by <B>pcre2_jit_compile()</B> before being
|
|
serialized, the JIT data is discarded and so is no longer available after a
|
|
save/restore cycle. You can, however, process a restored pattern with
|
|
<B>pcre2_jit_compile()</B> if you wish.
|
|
<A NAME="lbAG"> </A>
|
|
<H2>AUTHOR</H2>
|
|
|
|
|
|
<P>
|
|
<PRE>
|
|
Philip Hazel
|
|
University Computing Service
|
|
Cambridge, England.
|
|
</PRE>
|
|
|
|
<A NAME="lbAH"> </A>
|
|
<H2>REVISION</H2>
|
|
|
|
|
|
<P>
|
|
<PRE>
|
|
Last updated: 27 June 2018
|
|
Copyright (c) 1997-2018 University of Cambridge.
|
|
</PRE>
|
|
|
|
<P>
|
|
|
|
<HR>
|
|
<A NAME="index"> </A><H2>Index</H2>
|
|
<DL>
|
|
<DT id="1"><A HREF="#lbAB">NAME</A><DD>
|
|
<DT id="2"><A HREF="#lbAC">SAVING AND RE-USING PRECOMPILED PCRE2 PATTERNS</A><DD>
|
|
<DT id="3"><A HREF="#lbAD">SECURITY CONCERNS</A><DD>
|
|
<DT id="4"><A HREF="#lbAE">SAVING COMPILED PATTERNS</A><DD>
|
|
<DT id="5"><A HREF="#lbAF">RE-USING PRECOMPILED PATTERNS</A><DD>
|
|
<DT id="6"><A HREF="#lbAG">AUTHOR</A><DD>
|
|
<DT id="7"><A HREF="#lbAH">REVISION</A><DD>
|
|
</DL>
|
|
<HR>
|
|
This document was created by
|
|
<A HREF="/cgi-bin/man/man2html">man2html</A>,
|
|
using the manual pages.<BR>
|
|
Time: 00:05:50 GMT, March 31, 2021
|
|
</BODY>
|
|
</HTML>
|