Rich tables: header shading, long links, nested tables, cell colours

docs/architecture/02_pipeline.md

@@ -38,14 +38,14 @@ A source with `structurizr:` is replaced by Markdown files before anything else
 
 | Filter | Job | Formats |
 |---|---|---|
-| `listtable.lua` | The blocks of `[[LISTTABLE]]` cells move into their cells, without captions. In Word a hidden bookmark marks the table for the finishing | all, and site pages |
+| `listtable.lua` | The blocks of `[[LISTTABLE]]` cells move into their cells, without captions. Cell colours and alignment become a style (HTML), `\cellcolor` (PDF) or a hidden bookmark the Word finishing reads. In PDF a table in a cell becomes a `tabular`. In Word a hidden bookmark marks the table for the finishing | all, and site pages |
 | `admonitions.lua` | Callouts in both spellings become a box of their kind | all |
 | `pagebreak.lua` | Page break blocks become a page break, or a new slide | all |
 | `markers.lua` | `[[CAPTION:]]`, `[[STYLE:]]` and orientation markers. On slides the caption also becomes the alt text pandoc prints under a picture | PowerPoint, HTML, PDF |
 | `center-div.lua` | Centring divs become LaTeX centring | PDF |
 | `site.lua` | Admonitions or asides, figures as picture and caption, no repeated title | site pages |
 
-Word handles the markers itself after pandoc (`pandoc/docx_post.py`), because they become Word fields, sections and styles. `pandoc/docx_finish.py` merges the result into the template, fills the placeholders and applies the layout. It merges with docxcompose one list at a time, because docxcompose gives every list in one table the same numbering. Tables get the template's table style, centred at full width. An ordinary table with a header row is autofit, its header repeated and its rows kept whole. A `[[LISTTABLE]]` (found by its bookmark) or a table without a header keeps pandoc's column widths in a fixed layout. Its body rows may break across pages, and its pictures are made no wider than their cells. `pandoc/pptx_finish.py` keeps embedded fonts, fills the version token and the footer, and warns about crowded slides. The finished file is moved to its output path, and the build report records it with its checksum.
+Word handles the markers itself after pandoc (`pandoc/docx_post.py`), because they become Word fields, sections and styles. `pandoc/docx_finish.py` merges the result into the template, fills the placeholders and applies the layout. It merges with docxcompose one list at a time, because docxcompose gives every list in one table the same numbering. Before the merge each table of the Markdown gets a hidden bookmark, so the template's own tables are left as they are. Tables get the template's table style, centred at full width, with the header row's colour set on its cells. An ordinary table with a header row is autofit, its header repeated and its rows kept whole. A `[[LISTTABLE]]` (found by its bookmark) or a table without a header keeps pandoc's column widths in a fixed layout. Its body rows may break across pages, and its pictures are made no wider than their cells. `pandoc/pptx_finish.py` keeps embedded fonts, fills the version token and the footer, and warns about crowded slides. The finished file is moved to its output path, and the build report records it with its checksum.
 
 ## Around the documents
 

docs/guide/03_writing.md

@@ -191,39 +191,48 @@ Word uses the `Div.note` to `Div.caution` paragraph styles. The built-in referen
 
 ## Rich table cells
 
-A pipe table holds one line of text in each cell. For cells of several paragraphs, lists, merged cells or a diagram, put `[[LISTTABLE]]` on the line before the rows, and leave out the `|---|` line:
+A pipe table holds one line of text in each cell. For cells of several paragraphs, lists, merged cells, colours, a diagram or another table, put `[[LISTTABLE]]` on the line before the rows, and leave out the `|---|` line:
 
 ````markdown
 [[LISTTABLE: header, widths=8,52,40]]
 | Step | What to do | Check |
-| 1 | [Prepare the node:]{.underline}<br>Fit the batteries.<br>Before you go to site:<br>- The firmware is version {{version}}<br>- The node is registered<br>  - with its own certificate | The light flashes green. |
+| 1 | [Prepare the node:]{.underline}<br>Fit the batteries.<br>Before you go to site:<br>- The firmware is version {{version}}<br>- The node is registered<br>  - with its own certificate | {bg=E8F5E9 valign=middle} The light flashes green. |
 | 2 | Pair the node:<br>1. Hold the button for five seconds.<br>2. Choose the node in the app. | ^ |
 | 3 | The first reading goes this way:<br>[[BLOCK: reading]] | < |
+| 4 | Wire the terminals:<br>[[BLOCK: terminals]] | See the [sheet](https://example.com/sn-100/installation/terminal-wiring.pdf). |
 
 [[BLOCK: reading]]
 ```dot
 digraph reading { rankdir=LR; Sensor -> Gateway -> Broker; }
 ```
 [[ENDBLOCK]]
+
+[[BLOCK: terminals]]
+| Terminal | Wire |
+|---|---|
+| V+ | Red |
+[[ENDBLOCK]]
 ````
 
 - **Rows**: each row is one line that starts and ends with `|`, straight after the marker. `\|` is a pipe in the text.
 - **Header**: `header` makes the first row the header row. Without it every row is body text.
 - **Widths**: `widths=8,52,40` gives the relative widths of the columns. Without it the columns are the same width.
 - **Paragraphs and lists**: `<br>` starts a new paragraph in a cell. A paragraph that starts with `- ` is a bullet and one that starts with `1. ` a numbered item, and items next to each other make one list. Two spaces before the `- ` or `1. ` put the item inside the one above.
 - **Merged cells**: a cell of only `<` joins the cell on its left, and a cell of only `^` the cell above. Merged cells must make a rectangle, and a body cell cannot join the header row. `\<` and `\^` are the characters themselves.
-- **Blocks**: a cell paragraph `[[BLOCK: name]]` is replaced by the lines between `[[BLOCK: name]]` and `[[ENDBLOCK]]` elsewhere in the same file. A block can hold a diagram, a picture, code or a callout. A picture or diagram in a cell has no caption of its own: caption the table. Plain pictures also work inline, as `![alt](picture.png)`.
+- **Colour and alignment**: a cell that starts with `{bg=E8F5E9}` has that background colour, as six hex digits. `{valign=middle}` puts its text in the middle of the row, and `top` and `bottom` work too. Both can go in one `{...}`. In PDF a colour covers the first row of a cell merged down, and the alignment is not used.
+- **Blocks**: a cell paragraph `[[BLOCK: name]]` is replaced by the lines between `[[BLOCK: name]]` and `[[ENDBLOCK]]`, written elsewhere in the same file or in a file it includes, or the file that includes it. A block can hold a diagram, a picture, code, a callout or a table, including another `[[LISTTABLE]]`. A picture or diagram in a cell has no caption of its own: caption the table. Plain pictures also work inline, as `![alt](picture.png)`.
+- **Long links**: a link or picture too long for its column is written as a reference after the table, so it keeps the table readable. It is still the same link.
 - **Formatting**: bold, italic, links, `[text]{.underline}` and `[text]{custom-style="Name"}` work as anywhere else. A `custom-style` is a character style of the Word template. Pandoc makes one with no formatting if the template has none of that name, so add the styles you use to the template.
 
-py-doc turns the table into a pandoc grid table after includes, conditions and variables, so rich tables work in included files and take `{{variables}}`. A marker in a code block is left alone. In Word the columns keep the widths given, the rows may break across pages, and a picture is made no wider than its cell. Only a header row is repeated on each page and printed as a header. HTML, PDF and sites show the same table. On a slide pandoc shows merged cells as separate cells, bullets as plain lines and no pictures.
+py-doc turns the table into a pandoc grid table after includes, conditions and variables, so rich tables work in included files and take `{{variables}}`. A marker in a code block is left alone. In Word the columns keep the widths given, the rows may break across pages, and a picture is made no wider than its cell. Only a header row is repeated on each page and printed as a header. A table in a cell is formatted like any other. HTML, PDF and sites show the same table. On a slide pandoc shows merged cells as separate cells, bullets as plain lines and no pictures, colours or tables in cells.
 
 The build warns about a row with too many or too few cells, which is padded or cut. It warns about a merge that cannot be made, which is left empty, and a marker with no rows, which is left as written. It also warns about a block that is used but not defined, or defined but not used. A `[[BLOCK:]]` that is never closed with `[[ENDBLOCK]]` stops the build. `py-doc lint` and `py-doc check` report all of these with their line.
 
 Known limits:
 
-- A word wider than its column breaks the table, and the build says so. Widen the column or shorten the word.
+- A word wider than its column breaks the table, and the build says so. Widen the column or shorten the word. Links and pictures are shortened for you, but not a long bare address such as `<https://...>`.
 - Includes and conditions cannot go inside a cell, only around the whole table or inside a block.
-- A block belongs to its file: a table cannot use a block defined in another file.
+- A block is shared by a source file and the files it includes, not by the other sources of a document.
 
 ## Includes, conditions and variables
 

docs/guide/04_outputs.md

@@ -21,6 +21,7 @@ Pandoc writes the document with the template as its reference document, so the t
 - **Merging**: when a template is set, the result is merged into it. The body goes where `[[INSERTCONTENT]]` stands, or at the end if there is no marker. `[[LANDSCAPE]]`, `[[PORTRAIT]]`, `[[STYLE:Name]]` and `[[CAPTION:Figure]]` lines are applied afterwards.
 - **Placeholders**: `Replace-with-Title`, `-Author`, `-Version`, `-Document-Number`, `-Date`, `-Description` and `-Classification` are filled in the template body, tables, headers and footers. `placeholder_prefix` (default `Replace-with-`) and `marker` (default `[[INSERTCONTENT]]`) can be changed. A placeholder the document has no value for is emptied with a warning, and one that no property can fill (`Replace-with-Mystery`) is left and named in a warning.
 - **Heading numbers**: a template can number its heading styles (a `Heading 1` with list numbering). The numbers written in the headings would then show twice, so py-doc leaves them out: `## 2.1 Design` becomes `## Design`, numbered by Word. `heading_numbers: strip` or `keep` decides it for any template. See [Heading numbers](#heading-numbers).
+- **Tables** from the Markdown get the template's `Table Grid` style, centred at full width. The header row is bold white on the colour the style gives a header row, or navy, set on the cells themselves so that every program shows it. It repeats on each page. A picture in a cell is made no wider than its column, and a table in a cell is formatted the same way. The template's own tables, such as a title page table of document facts, keep the look the template gives them.
 - **Contents list**: `docx: {toc: true}` adds a table of contents to the depth of `toc_depth` (front matter). Word fills in the page numbers when the file is opened and asks to update fields.
 - **Document properties** (title, author, subject, keywords, category, comments) are capped at Word's 255 characters, with a warning when a value is cut.
 - **Code** is highlighted with the built-in theme.

docs/requirements/01_functional.md

@@ -35,7 +35,7 @@ title: Functional requirements
 | FR-026 | Fenced blocks of Mermaid, PlantUML, Graphviz, ditaa, the blockdiag family, WaveDrom, bytefield-svg, Pikchr, railroad, WireViz, Vega and Vega-Lite shall become pictures in every output. | Must | Met |
 | FR-027 | A diagram that cannot be drawn shall stop the build with the file, the line and the tool's message. `--keep-going` shall list every failure. | Must | Met |
 | FR-028 | `diagrams.mermaid` in `py-doc.yml` shall give every Mermaid diagram of documents, sites and posters the same Mermaid settings, under the diagram's own. | Could | Met |
-| FR-029 | `[[LISTTABLE]]` followed by `\| ... \|` rows shall make a table with an optional header row and relative column widths. Its cells shall hold several paragraphs (`<br>`), bullet and numbered lists (`- `, `1. `) nested by indentation, cells merged across (`<`) and down (`^`), and blocks such as diagrams (`[[BLOCK:]]`). In Word its body rows may break across pages, only a header row shall be formatted and repeated as one, and a picture shall fit its cell. These mistakes shall be warnings that name the file and line: a wrong number of cells, an impossible merge, a marker with no rows, an undefined block. A block never closed shall be an error. | Should | Met |
+| FR-029 | `[[LISTTABLE]]` followed by `\| ... \|` rows shall make a table with an optional header row and relative column widths. Its cells shall hold several paragraphs (`<br>`), bullet and numbered lists (`- `, `1. `) nested by indentation, and blocks such as diagrams and tables (`[[BLOCK:]]`). Cells shall merge across (`<`) and down (`^`), and take a colour and alignment (`{bg=RRGGBB valign=middle}`). A block may be in the file or in a file it includes. A link too long for its column shall become a reference. In Word its body rows may break across pages, only a header row shall be formatted and repeated as one, and a picture shall fit its cell. These mistakes shall be warnings that name the file and line: a wrong number of cells, an impossible merge, a marker with no rows, an undefined block. A block never closed shall be an error. | Should | Met |
 
 ## Outputs
 
@@ -51,6 +51,7 @@ title: Functional requirements
 | FR-047 | `py-doc template` shall check, list, compare and count the styles of Word and PowerPoint templates. | Should | Met |
 | FR-048 | `py-doc docx` and `py-doc slides` shall convert single files and folders without a `py-doc.yml`. | Must | Met |
 | FR-049 | Where the output numbers its headings itself, the section numbers written in headings shall be left out, so no section is numbered twice. That is a Word template with numbered heading styles, or numbered PDF and HTML sections. A setting shall force or prevent it. | Must | Met |
+| FR-050 | In Word, the header row of a table from the Markdown shall show in every program, and its pictures shall be no wider than their column. The template's own tables shall keep their look. | Must | Met |
 
 ## Importing
 

docs/requirements/04_traceability.md

@@ -13,7 +13,7 @@ title: Traceability
 | FR-008 | `migrate.py` | `test_migrate.py` |
 | FR-020 to FR-025 | `directives.py`, `pandoc/resources/*.lua`, `pandoc/docx_post.py` | `test_directives.py`, `test_pandoc.py`, `test_word_control.py`, `test_word_layout.py` |
 | FR-026, FR-027, QR-080 | `diagrams/` | `test_diagrams.py` |
-| FR-040 to FR-049 | `pandoc/engine.py`, `pandoc/reference.py`, `pandoc/docx_finish.py`, `pandoc/pptx_finish.py`, `naming.py`, `report.py`, `template_check.py`, `convert.py`, `_cli.py` | `test_pandoc.py`, `test_pptx_pandoc.py`, `test_report.py`, `test_template_check.py`, `test_cli.py`, `test_slides_cli.py`, `test_convert.py`, `test_examples.py`, `test_heading_numbers.py` |
+| FR-040 to FR-050 | `pandoc/engine.py`, `pandoc/reference.py`, `pandoc/docx_finish.py`, `pandoc/pptx_finish.py`, `naming.py`, `report.py`, `template_check.py`, `convert.py`, `_cli.py` | `test_pandoc.py`, `test_pptx_pandoc.py`, `test_report.py`, `test_template_check.py`, `test_cli.py`, `test_slides_cli.py`, `test_convert.py`, `test_examples.py`, `test_heading_numbers.py`, `test_listtable.py` |
 | FR-060, FR-062 to FR-064 | `docx_import.py`, `import_cli.py` | `test_docx_import.py` |
 | FR-061 | `pptx_import.py` | `test_pptx_import.py` |
 | FR-080 to FR-086, QR-081 | `c4/` | `test_c4.py` |

examples/docs/guide/03_operations.md

@@ -22,16 +22,24 @@ This paragraph is shown to everyone except staff.
 
 ## Commissioning
 
-A rich table holds paragraphs, lists, merged cells and diagrams in its cells. `<br>` starts a new paragraph, `- ` a bullet and `1. ` a numbered item, and two spaces put an item inside the one above. A cell of `<` joins the cell on its left, and `^` the cell above.
+A rich table holds paragraphs, lists, merged cells, diagrams and other tables in its cells. `<br>` starts a new paragraph, `- ` a bullet and `1. ` a numbered item, and two spaces put an item inside the one above. A cell of `<` joins the cell on its left, and `^` the cell above. `{bg=E8F5E9 valign=middle}` at the start of a cell gives it a colour and an alignment.
 
 [[CAPTION:Table]] Commissioning steps
 
 [[LISTTABLE: header, widths=8,52,40]]
 | Step | What to do | Check |
-| 1 | [Prepare the node:]{.underline}<br>Fit the batteries and close the enclosure.<br>Before you go to site:<br>- The firmware is version {{version}}<br>- The node is registered on the broker<br>  - with its own client certificate | The light flashes green once. |
+| 1 | [Prepare the node:]{.underline}<br>Fit the batteries and close the enclosure.<br>Before you go to site:<br>- The firmware is version {{version}}<br>- The node is registered on the broker<br>  - with its own client certificate | {bg=E8F5E9 valign=middle} The light flashes green once. |
 | 2 | Pair the node over Bluetooth:<br>1. Hold the button for five seconds.<br>2. Choose the node in the app. | ^ |
 | 3 | The first reading goes this way:<br>[[BLOCK: reading]] | < |
-| 4 | [Report a node that publishes nothing to support.]{custom-style="Highlight Yellow"} | < |
+| 4 | Wire the terminals:<br>[[BLOCK: terminals]] | See the [installation sheet](https://example.com/sn-100/installation/terminal-wiring-sheet.pdf). |
+| 5 | [Report a node that publishes nothing to support.]{custom-style="Highlight Yellow"} | < |
+
+[[BLOCK: terminals]]
+| Terminal | Wire |
+|---|---|
+| V+ | Red |
+| 0 V | Black |
+[[ENDBLOCK]]
 
 [[BLOCK: reading]]
 ```dot

examples/templates/make_templates.py

@@ -118,10 +118,15 @@ def highlight_styles(document) -> None:
             document.styles.add_style(name, WD_STYLE_TYPE.CHARACTER).font.highlight_color = colour
 
 
+def remove_sample(document) -> None:
+    """Take out pandoc's sample text and sample table, which its reference document starts with."""
+    for block in [*document.paragraphs, *document.tables]:
+        block._element.getparent().remove(block._element)
+
+
 def title_page(document) -> None:
     """Logo, title, description and a table of document facts, all filled in from the front matter."""
-    for paragraph in list(document.paragraphs):
-        paragraph._element.getparent().remove(paragraph._element)  # pandoc's sample text
+    remove_sample(document)
     styles = document.styles
     logo = document.add_paragraph(style=find(styles, "Normal"))
     logo.alignment = WD_ALIGN_PARAGRAPH.CENTER
@@ -184,8 +189,7 @@ def datasheet_docx() -> None:
     ``py-doc.yml`` names them with ``placeholder_prefix`` and ``marker``."""
     document = Document(io.BytesIO(pandoc_file("reference.docx")))
     set_styles(document)
-    for paragraph in list(document.paragraphs):
-        paragraph._element.getparent().remove(paragraph._element)
+    remove_sample(document)
     document.add_paragraph("@@Title", style="Title")
     document.add_paragraph("@@Description", style="Lead Paragraph")
     document.add_paragraph("Version @@Version, @@Date", style="Normal")

examples/templates/report.tex

@@ -32,6 +32,7 @@
 \usepackage{adjustbox}
 \usepackage{longtable,booktabs,array,calc,multirow} % multirow: cells merged down, in [[LISTTABLE]] tables
 \usepackage{xcolor}
+\usepackage{colortbl} % \cellcolor: cell colours, in [[LISTTABLE]] tables
 \usepackage{amsmath}  % \text, align and the other commands pandoc writes for maths
 \usepackage{amssymb}  % \boxtimes and \square, which pandoc writes for task lists
 % pandoc writes ~~struck out~~ as \st{} or \sout{}, and underlined text as \ul{} or \uline{}

examples/templates/report_Draft.tex

@@ -33,6 +33,7 @@
 \usepackage{adjustbox}
 \usepackage{longtable,booktabs,array,calc,multirow} % multirow: cells merged down, in [[LISTTABLE]] tables
 \usepackage{xcolor}
+\usepackage{colortbl} % \cellcolor: cell colours, in [[LISTTABLE]] tables
 \usepackage{amsmath}  % \text, align and the other commands pandoc writes for maths
 \usepackage{amssymb}  % \boxtimes and \square, which pandoc writes for task lists
 % pandoc writes ~~struck out~~ as \st{} or \sout{}, and underlined text as \ul{} or \uline{}