40 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
6180e3a051
doc: wikiparser: Fix issue with running parser outside doc/ dir
Use a path relative to the script path when checking for existence of manual
pages.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 10:59:08 -07:00
James Valleroy
085e3ffef3
wikiparser: Exit with return value 1 on test failure
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 10:59:02 -07:00
Sunil Mohan Adapa
04a0fd7b78
doc: wikiparser: Show icons with full size
Setting the height to 26 in the XML file results in HTML getting a 16px size.
16x16 is original size of all the icons. This was first proposed with a size of
20 in the original wikiparser MR.

Tests:

- Affected sections/pages of the manual are: Firewall, Hardware and VirtualBox.
Confirm by diffing with manual generated without the code changes.

- Open all affected pages and the full manual page and confirm that images sizes
are 16px (original size).

- Check all affected pages and the full manual page in FreedomBox interface and
confirm that images sizes are 16px (original size).

- Check all affected sections in the generated PDF.

- Repeat tests for English and Spanish manuals.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-09-15 15:23:27 -04:00
Sunil Mohan Adapa
57c60d9ac6
doc: wikiparser: Use icons from the icons directory
Currently some icons are duplicated between the images/ directory and
images/icons/ directory. De-duplicate and only use the icons from images/icons
directory.

Tests:

- Affected sections/pages of the manual are: Firewall, Hardware and VirtualBox.
Confirm by diffing with manual generated without the code changes.

- Open all affected pages and the full manual page and confirm that images are
loading properly.

- Check all affected pages and the full manual page in FreedomBox interface and
confirm that images are loading properly.

- Check all affected sections in the generated PDF.

- Repeat tests for English and Spanish manuals.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-09-15 15:23:24 -04:00
Sunil Mohan Adapa
95c99bf3ce
doc: wikiparser: Reduce build verbosity
- Make begin and end markers the default values in the wikiparser arguments so
that they don't have to passed in every time.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:31 -07:00
Sunil Mohan Adapa
099d877b8f
doc: wikiparser: Make it easier to run with a #! at the top
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:28 -07:00
Sunil Mohan Adapa
95452c1313
doc: wikiparser: Fix issue with some URL containing dup. lang part
When getting context in included images, the original context might already have
a language component. Don't add it again if it does.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:25 -07:00
Sunil Mohan Adapa
0296cfc97f
doc: wikiparser: Fix a minor flake8 issue
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:55:50 -07:00
Fioddor Superconcentrado
42037b8330
doc: wikiparser: Resolve URLs for locally available pages
Closes: #1793

resolve_url() now checks for local help pages and shortcuts the links that point
to them to the local copy. Unit doctests have been included also. Expectations
of 2 preexisting tests resulted impacted by the new logic (shortcutted links)
and were adapted accordingly. New doctests unveiled some rough edge of the
previous logic, so the processing of keyword-protocolled links was fine-tuned to
pollish them.

Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Fixed minor indentation, flake8 issues, styling, comments]
[sunil: Use file exists checking instead of iterating directory]
[sunil: Stricter checking using regex for possible local URLs]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:55:15 -07:00
James Valleroy
ce642e3e87
wikiparser: Don't render Admonition with style comment
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:13:28 -04:00
Sunil Mohan Adapa
72ca92224d
doc: wikiparser: Add note about some incorrect links
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:38 -04:00
Sunil Mohan Adapa
7c7f71224f
doc: wikiparser: Incorporate post processing fixes
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:33 -04:00
Sunil Mohan Adapa
6c8c6758d9
doc: wikiparser: Format text inside admonitions properly
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:28 -04:00
Sunil Mohan Adapa
7ae46e4ffa
doc: wikiparser: Handle markup inside italic/bold markup
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:26 -04:00
Sunil Mohan Adapa
8742547a13
doc: wikiparser: Fix issue with parsing inline code blocks
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:23 -04:00
Sunil Mohan Adapa
49d554fb0c
doc: wikiparser: Improve links relative to included files
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:21 -04:00
Sunil Mohan Adapa
c522583ecd
doc: wikiparser: Fix some flake8 warnings
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:19 -04:00
Sunil Mohan Adapa
f54e32ae7e
doc: wikiparser: Handle empty table cells
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:16 -04:00
Sunil Mohan Adapa
4448b5baa8
doc: wikiparser: Set icon dimensions to old values (temporarily)
- For easier diff

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:14 -04:00
Sunil Mohan Adapa
feb81b8fb6
doc: wikiparser: Retain the text for icons
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:10 -04:00
Sunil Mohan Adapa
5f947ba1e7
doc: wikiparser: Sort imagedata properties
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:07 -04:00
Sunil Mohan Adapa
cafbdaab19
doc: wikiparser: Place anchors inside paragraphs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:05 -04:00
Sunil Mohan Adapa
f3dee93a61
doc: wikiparser: Strip spaces from attachment's text
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:03 -04:00
Sunil Mohan Adapa
5aa31266e2
doc: wikiparser: Allow lists to started with just spaces
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:00 -04:00
Sunil Mohan Adapa
4fd5d0e73b
doc: wikiparser: Handle wiki links starting with a /
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:58 -04:00
Sunil Mohan Adapa
c7afe8c688
doc: wikiparser: Assign text to URLs that don't provide them
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:55 -04:00
Sunil Mohan Adapa
0bb8dc68fe
doc: wikiparser: Handle existing # in links, don't append again
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:53 -04:00
Sunil Mohan Adapa
d760ee004f
doc: wikiparser: Preserve spaces during parsing and generation
- Matches the original XML more accurately and simplifies some parsing.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:50 -04:00
Sunil Mohan Adapa
aa27037c71
doc: wikiparser: Resolve relative URLs
- Handle links such as DebianBug:, DebianPkg:, AliothList:, attachment:, irc://,
mailto://

- Handle relative links that start with ../

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:48 -04:00
Sunil Mohan Adapa
98ba279850
doc: wikiparser: Fix parsing URLs, simplify plain text parsing
- Parse Wiki words like MoinMoin.

- Handle . : etc. at the end of the links properly.

- Use regular expressions to simplify parsing plain text.

- Strip spaces in link targets and text.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:45 -04:00
Sunil Mohan Adapa
3aca4cd164
doc: wikiparser: Allow empty lines between list items
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:43 -04:00
Sunil Mohan Adapa
9bb518cc76
doc: wikiparser: Parse content inside a comment
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:41 -04:00
Sunil Mohan Adapa
1f94ac4bce
doc: wikiparser: Handle more paragraph breakers
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:38 -04:00
Sunil Mohan Adapa
e5c51e9bb4
doc: wikiparser: Handle table of contents macro without parenthesis
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:36 -04:00
Sunil Mohan Adapa
64fedf1f32
doc: wikiparser: Generate colspec for tables
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:33 -04:00
Sunil Mohan Adapa
e947e99af3
doc: wikiparser: When processing single pages, ignore header/footer
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:30 -04:00
Sunil Mohan Adapa
4ef2244af4
doc: wikiparser: Fix attachment URLs in regular links
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:28 -04:00
Sunil Mohan Adapa
40f853278b
doc: wikiparser: Handle processing instructions
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:25 -04:00
James Valleroy
806db903cf
wikiparser: Fix spaces, multi-line, languages, icons
- Allow for additional languages to be added
- Fix unwanted spaces in paragraphs
- Handle multi-line ListItems
- Handle icons
- Add line break before inline image in list

Based on patches by Fioddor Superconcentrado <fiodor@gmail.com>.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-08-28 12:09:05 -04:00
James Valleroy
5b0335a2fa
doc: Add moinmoin wiki parser
- Handle internal links
- Handle includes
- Handle generic categories and includes
- Handle embedded links
- Handle lists
- Handle admonitions and code blocks
- Generate docbook from wiki
- Use argparse
- Generate <itemizedlist> for lists
- Generate docbook article for testing
- Escape content strings
- Create nested <section>s for headings
- Handle links with empty text
- Generate comments as <remark> within <para>
- Add space at end of paragraph
- Ignore Category when generating
- Generate comment for horizontal rule
- Add missing structure to tables
- Don't generate End Include
- Handle Includes
- Handle BEGIN_INCLUDE
- Process begin and end markers when including file
- Handle embedded attachments
- Add a helper script to test all pages
- Generate docbook for admonitions
- Fix include context
- Don't generate for table of contents
- Ignore lines starting with ##
- Ignore space in comparison test
- Use <para> around listitem content
- Wrap code text in <screen>
- Only split plain text for double underscore
- Generate warning for smaller text
- Handle relative links to wiki
- Fix handling of recursive includes
- Parse URLs out of text
- Handle image attachment parameters
- Use SPDX license identifier
- Handle height property for images
- Strip whitespace when using startswith
- Add cols property to tgroup
- Handle spaces at start and end of multi-line CodeText
- Fix page names
- Make consistent in generating <para>s
- Rewrite list handling to support nested lists properly
- Handle embedded attachments inside links
- Handle links within emphasis
- Handle wiki links to InstallingDebianOn/
- Handle line breaks
- Use paragraphs in Admonitions.
- Handle Anchors
- Don't escape code text
- Fix stopping paragraph when list begins
- Fix characters dropped after PlainText
- Fix finding http and https links in text
- Ignore example URLs containing < or >
- Pass context to every call of to_docbook()
- Render CodeText in paragraph as simply <code>
- Ensure only tablestyle is removed from table cells
- Handle line breaks within table cells
- Handle text alignment style within table cell
- Only split plain text for double brace or double bracket
- Ignore lines with only whitespace for paragraphs
- Make tests verbose if no input files givena

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-08-28 12:08:58 -04:00