Syntax highlighting#

The standard actions used in the bundled languages can be mapped to colors and other text formatting properties, so text in any language can be highlighted in a pretty way, to improve readability.

This is done by combining a Formatter with a Theme.

A Formatter iterates over the tokens in a selected range and yields FormatRange tuples, describing how a certain piece of text should be formatted. A format range consists of a starting position pos, an ending position end and a format textformat.

This textformat is provided by the theme, which maps a standard action to a TextFormat, and converted to something the formatter can use using a factory function that is specified when creating the formatter.

A TextFormat provided by the theme is a simple data object with attributes that define text properties, such as color, font, decoration etc. The default Theme implementation reads these properties from a CSS (Cascading Style Sheets) file. Some CSS themes are provided, in the themes directory.

You can implement any kind of formatting by creating a Formatter with your factory function, and then iterating over format_ranges().

Optionally, you can inherit from Formatter to implement useful other methods. In parce.out there are some modules containing often used formatting utilities.

Creating HTML output#

Using the parce.out.html module, it is easy to convert tokenized text to HTML. Here is an example. Let’s say we’ve got a document containing some XML text:

>>> import parce
>>> doc = parce.Document(parce.find("xml"), '''<xml attr="value">text</xml>\n''')

We create a cursor that selects all text (a formatter can also just format a selection of the text):

>>> cur = parce.Cursor(doc, 0, None)

We load a theme, using the parce/themes/dark.css included CSS theme:

>>> theme = parce.theme_by_name('dark')

And create an HTML formatter:

>>> from parce.out.html import HtmlFormatter
>>> f = HtmlFormatter(theme)

Now we call the formatter to format the selected part of the document:

>>> print(f.full_html(cur))
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <div class="parce">
<pre style="white-space: pre; background-color: #000000; color: #fffff0; fon
t-family: monospace;">&lt;<span style="color: #87cefa; font-weight: bold;">x
ml</span> <span style="color: #1e90ff;">attr</span>=<span style="color: #cd5
c5c;">"value"</span>&gt;text&lt;/<span style="color: #87cefa; font-weight: b
old;">xml</span>&gt;
</pre>
    </div>
  </body>
</html>

Creating your own themes#

The easiest way to create your own theme is by copying default.css or _template.css in the themes/ directory to a new file and start editing that.

CSS properties#

The following subset of CSS properties is supported by the default TextFormat used by the theming engine:

Property:	Supported values:
`color`	named CSS color (like `antiquewhite`), hex color (like `#02030A`) with optional alpha value, `rgb()` and `rgba()` colors.
`background-color`	same as `color`
`background`	only colors are supported, same as `color`
`caret-color`	same as `color`
`text-decoration-color`	same as `color`
`text-decoration-line`	one or more of `underline`, `overline`, `line-through` and `none`
`text-decoration-style`	one of `solid`, `double`, `dotted`, `dashed` or `wavy`
`text-decoration`	in order a color, line, and style value
`font-family`	one or more generic or quoted font names; generic names are: `serif`, `sans-serif`, `monospace`, `cursive`, `fantasy`, `system-ui`, `math`, `emoji` and `fangsong`.
`font-kerning`	one of `auto`, `normal` or `none`
`font-size`	one of `xx-small`, `x-small`, `small`, `medium`, `large`, `x-large`, `xx-large`, `xxx-large`, `larger`, `smaller` or a numeric value, optionally with a `%` or unit like `pt`, `em` etc.
`font-stretch`	one of `ultra-condensed`, `extra-condensed`, `condensed`, `semi-condensed`, `semi-expanded`, `expanded`, `extra-expanded` or `ultra-expanded`, or a numerical value with a `%`.
`font-style`	`normal`, `italic`, or `oblique` with an optional slant value and unit
`font-variant-caps`	one of `normal`, `small-caps`, `all-small-caps`, `petite-caps`, `all-petite-caps`, `unicase`, `titling-caps`
`font-variant-position`	one of `normal`, `sub`, or `super`
`font-weight`	one of `normal`, `bold`, `lighter`, `bolder`, or a number
`font`	all of the above `font-*` properties, or one of: `caption`, `icon`, `menu`, `message-box`, `small-caption`, `status-bar`

Note

It is possible that not all formatters support all properties. For example Qt5’s QTextCharFormat does not support double underline.

CSS classes#

To determine the style properties to use for a token, the token’s action (which must be a standard action) is mapped to one or more CSS classes. This is described in The theme module, under “Mapping actions to CSS classes.” The matching CSS rules are then combined to determine the actual style properties to use for the action.

All rules should have a .parce ancestor class selector, so that the theme css file can directly be used in HTML (where tokens are mapped to class names, e.g. using the SimpleHtmlFormatter), without much chance that other parts of a web page’s style are clobbered by the parce css file, for example:

.parce
.comment {
    color: dimgray;
    font-family: serif;
    font-style: italic;
}

This maps the Comment standard action to these color and font settings.

General classes#

There are some special classes that define other style aspects than that of individual tokens:

CSS Selector	defines properties to use for:
`.parce`	the text view or block as a whole; e.g. a text editor window, or an HTML `<pre>` block. A text editor is free to ignore font settings.
`.parce.current-line`	the line the cursor is in (only background probably makes sense)
`.parce::selection`	text selected by the user (also works in straight HTML in a modern browser)
`.parce.current-line:focus`	the current line when the window has focus
`.parce::selection:focus`	selected text when the window has focus
`.parce:disabled`	the text editor widget when it is disabled (i.e. the user can’t interact with it). If a text editor supports this at all, probably only the changed colors will be used (via a widget’s palette), not the font.
`.parce.current-line:disabled`	the current line when the text widget is disabled
`.parce::selection:disabled`	selected text when the text widget is disabled

Editor features#

The following CSS classes are not used by the parce formatter itself, but they can be used to define text editor styles so that they relate with the theme (other classes might be designed as well for custom usage of parce themes):

CSS Selector	defines properties to use for:
`.parce.leading-whitespace`	highlighting leading whitespace, if desired.
`.parce.trailing-whitespace`	highlighting trailing whitespace, if desired.
`.parce.eol-marker`	drawing an “end-of-line” marker, if desired.
`.parce.margin-linenumbers`	the region displaying line numbers.
`.parce.margin-folding`	the region displaying folding markers.
`.parce.mark-line`	a (book)marked line (at least background should be defined)
`.parce.mark-search`	search results in the text (at least background should be defined)
`.parce.mark-error`	a line marked as containing e.g. a compile error in the text (at least background should be defined)
`.parce.brace-match`	a matching bracket or parenthesis, etc.
`.parce.brace-nomatch`	a bracket or parenthesis that has no valid match.

Using multiple themes together#

Suppose you want to highlight tokens from embedded pieces of a different language with a different theme. E.g. you have a document containing HTML markup and want to highlight embedded CSS with a different color theme.

To do this, create a formatter and then add other themes for specific languages:

>>> import parce
>>> doc = parce.Document(parce.find("html"), '''
... <html>
... <head>
... <style type="text/css">
... h2 {
...     color: green;
... }
... </style>
... </head>
... </html>
... ''')
>>> from parce.out.html import HtmlFormatter
>>> f = HtmlFormatter(parce.theme_by_name('default'))
>>> f.add_theme(parce.find("css").language, parce.theme_by_name('dark'))
>>> print(f.full_html(parce.Cursor(doc, 0, None)))
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <div class="parce">
<pre style="white-space: pre; background-color: #fffff0; color: #000000; font-family: monospace;">
&lt;<span style="color: #00008b; font-weight: bold;">html</span>&gt;
&lt;<span style="color: #00008b; font-weight: bold;">head</span>&gt;
&lt;<span style="color: #00008b; font-weight: bold;">style</span> <span style="color: #1e90ff;">type</span>=<span style="color: #b22222;">"text/css"</span>&gt;
<span style="color: #87cefa; font-weight: bold;">h2</span> <span style="font-weight: bold;">{</span>
    <span style="color: #4169e1; font-weight: bold;">color</span>: <span style="color: #2e8b57;">green</span>;
<span style="font-weight: bold;">}</span>
&lt;/<span style="color: #00008b; font-weight: bold;">style</span>&gt;
&lt;/<span style="color: #00008b; font-weight: bold;">head</span>&gt;
&lt;/<span style="color: #00008b; font-weight: bold;">html</span>&gt;
</pre>
    </div>
  </body>
</html>

We used the default theme as default theme, and the dark theme for stuff that’s parsed by the CSS language.

In your browser, the resulting HTML-formatted text looks like this:

HTML

<html>
<head>
<style type="text/css">
h2 {
    color: green;
}
</style>
</head>
</html>

This example is not particularly beautiful, because the two themes are not really related; the css colors are quite light, because they expect a dark background. By default, the background of embedded language themes is not used. To force the formatter to use the default background color of embedded themes, add them to the formatter with add_baseformat = True:

>>> f.add_theme(parce.find("css").language, parce.theme_by_name('dark'), True)
>>> print(f.full_html(parce.Cursor(doc, 0, None)))
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8"/>
  </head>
  <body>
    <div class="parce">
<pre style="white-space: pre; background-color: #fffff0; color: #000000; font-family: monospace;">
&lt;<span style="color: #00008b; font-weight: bold;">html</span>&gt;
&lt;<span style="color: #00008b; font-weight: bold;">head</span>&gt;
&lt;<span style="color: #00008b; font-weight: bold;">style</span> <span style="color: #1e90ff;">type</span>=<span style="color: #b22222;">"text/css"</span>&gt;<span style="background-color: #000000; color: #fffff0; font-family: monospace;">
</span><span style="background-color: #000000; color: #87cefa; font-family: monospace; font-weight: bold;">h2</span><span style="background-color: #000000; color: #fffff0; font-family: monospace;"> </span><span style="background-color: #000000; color: #fffff0; font-family: monospace; font-weight: bold;">{</span><span style="background-color: #000000; color: #fffff0; font-family: monospace;">
    </span><span style="background-color: #000000; color: #4169e1; font-family: monospace; font-weight: bold;">color</span><span style="background-color: #000000; color: #fffff0; font-family: monospace;">: </span><span style="background-color: #000000; color: #2e8b57; font-family: monospace;">green</span><span style="background-color: #000000; color: #fffff0; font-family: monospace;">;
</span><span style="background-color: #000000; color: #fffff0; font-family: monospace; font-weight: bold;">}</span>
&lt;/<span style="color: #00008b; font-weight: bold;">style</span>&gt;
&lt;/<span style="color: #00008b; font-weight: bold;">head</span>&gt;
&lt;/<span style="color: #00008b; font-weight: bold;">html</span>&gt;
</pre>
    </div>
  </body>
</html>

When a theme is added to a formatter with add_baseformat = True, two things are done: 1) all unparsed text (text between tokens) is formatted using the baseformat of the embedded theme, and 2) all textformats of the embedded theme are combined with the theme’s baseformat.

The output looks like:

HTML

<html>
<head>
<style type="text/css">
h2 {
    color: green;
}
</style>
</head>
</html>

Of course the dark and default themes do not look good at all when used together, but this example shows that you, with well-designed themes and language definitions, can create sophisticated highlighting and code formatting with parce.