48 Commits

Author SHA1 Message Date
Sunil Mohan Adapa
6180e3a051
doc: wikiparser: Fix issue with running parser outside doc/ dir
Use a path relative to the script path when checking for existence of manual
pages.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 10:59:08 -07:00
James Valleroy
085e3ffef3
wikiparser: Exit with return value 1 on test failure
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-10-30 10:59:02 -07:00
Sunil Mohan Adapa
04a0fd7b78
doc: wikiparser: Show icons with full size
Setting the height to 26 in the XML file results in HTML getting a 16px size.
16x16 is original size of all the icons. This was first proposed with a size of
20 in the original wikiparser MR.

Tests:

- Affected sections/pages of the manual are: Firewall, Hardware and VirtualBox.
Confirm by diffing with manual generated without the code changes.

- Open all affected pages and the full manual page and confirm that images sizes
are 16px (original size).

- Check all affected pages and the full manual page in FreedomBox interface and
confirm that images sizes are 16px (original size).

- Check all affected sections in the generated PDF.

- Repeat tests for English and Spanish manuals.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-09-15 15:23:27 -04:00
Sunil Mohan Adapa
57c60d9ac6
doc: wikiparser: Use icons from the icons directory
Currently some icons are duplicated between the images/ directory and
images/icons/ directory. De-duplicate and only use the icons from images/icons
directory.

Tests:

- Affected sections/pages of the manual are: Firewall, Hardware and VirtualBox.
Confirm by diffing with manual generated without the code changes.

- Open all affected pages and the full manual page and confirm that images are
loading properly.

- Check all affected pages and the full manual page in FreedomBox interface and
confirm that images are loading properly.

- Check all affected sections in the generated PDF.

- Repeat tests for English and Spanish manuals.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-09-15 15:23:24 -04:00
Sunil Mohan Adapa
95c99bf3ce
doc: wikiparser: Reduce build verbosity
- Make begin and end markers the default values in the wikiparser arguments so
that they don't have to passed in every time.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:31 -07:00
Sunil Mohan Adapa
099d877b8f
doc: wikiparser: Make it easier to run with a #! at the top
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:28 -07:00
Sunil Mohan Adapa
95452c1313
doc: wikiparser: Fix issue with some URL containing dup. lang part
When getting context in included images, the original context might already have
a language component. Don't add it again if it does.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:56:25 -07:00
Sunil Mohan Adapa
0296cfc97f
doc: wikiparser: Fix a minor flake8 issue
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:55:50 -07:00
Fioddor Superconcentrado
42037b8330
doc: wikiparser: Resolve URLs for locally available pages
Closes: #1793

resolve_url() now checks for local help pages and shortcuts the links that point
to them to the local copy. Unit doctests have been included also. Expectations
of 2 preexisting tests resulted impacted by the new logic (shortcutted links)
and were adapted accordingly. New doctests unveiled some rough edge of the
previous logic, so the processing of keyword-protocolled links was fine-tuned to
pollish them.

Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[sunil: Fixed minor indentation, flake8 issues, styling, comments]
[sunil: Use file exists checking instead of iterating directory]
[sunil: Stricter checking using regex for possible local URLs]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-09-09 01:55:15 -07:00
James Valleroy
9a4c6a78a3
test-wikiparser: Remove fixes.xslt step
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:14:00 -04:00
James Valleroy
ce642e3e87
wikiparser: Don't render Admonition with style comment
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:13:28 -04:00
Sunil Mohan Adapa
4c78bcf89c
doc: Update the test script for wikiparser
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:40 -04:00
Sunil Mohan Adapa
72ca92224d
doc: wikiparser: Add note about some incorrect links
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:38 -04:00
Sunil Mohan Adapa
7c7f71224f
doc: wikiparser: Incorporate post processing fixes
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:33 -04:00
Sunil Mohan Adapa
09dd1e89e4
doc: Drop post processor as it is not needed anymore
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:30 -04:00
Sunil Mohan Adapa
6c8c6758d9
doc: wikiparser: Format text inside admonitions properly
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:28 -04:00
Sunil Mohan Adapa
7ae46e4ffa
doc: wikiparser: Handle markup inside italic/bold markup
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:26 -04:00
Sunil Mohan Adapa
8742547a13
doc: wikiparser: Fix issue with parsing inline code blocks
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:23 -04:00
Sunil Mohan Adapa
49d554fb0c
doc: wikiparser: Improve links relative to included files
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:21 -04:00
Sunil Mohan Adapa
c522583ecd
doc: wikiparser: Fix some flake8 warnings
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:19 -04:00
Sunil Mohan Adapa
f54e32ae7e
doc: wikiparser: Handle empty table cells
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:16 -04:00
Sunil Mohan Adapa
4448b5baa8
doc: wikiparser: Set icon dimensions to old values (temporarily)
- For easier diff

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:14 -04:00
Sunil Mohan Adapa
feb81b8fb6
doc: wikiparser: Retain the text for icons
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:10 -04:00
Sunil Mohan Adapa
5f947ba1e7
doc: wikiparser: Sort imagedata properties
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:07 -04:00
Sunil Mohan Adapa
cafbdaab19
doc: wikiparser: Place anchors inside paragraphs
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:05 -04:00
Sunil Mohan Adapa
f3dee93a61
doc: wikiparser: Strip spaces from attachment's text
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:03 -04:00
Sunil Mohan Adapa
5aa31266e2
doc: wikiparser: Allow lists to started with just spaces
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:10:00 -04:00
Sunil Mohan Adapa
4fd5d0e73b
doc: wikiparser: Handle wiki links starting with a /
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:58 -04:00
Sunil Mohan Adapa
c7afe8c688
doc: wikiparser: Assign text to URLs that don't provide them
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:55 -04:00
Sunil Mohan Adapa
0bb8dc68fe
doc: wikiparser: Handle existing # in links, don't append again
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:53 -04:00
Sunil Mohan Adapa
d760ee004f
doc: wikiparser: Preserve spaces during parsing and generation
- Matches the original XML more accurately and simplifies some parsing.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:50 -04:00
Sunil Mohan Adapa
aa27037c71
doc: wikiparser: Resolve relative URLs
- Handle links such as DebianBug:, DebianPkg:, AliothList:, attachment:, irc://,
mailto://

- Handle relative links that start with ../

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:48 -04:00
Sunil Mohan Adapa
98ba279850
doc: wikiparser: Fix parsing URLs, simplify plain text parsing
- Parse Wiki words like MoinMoin.

- Handle . : etc. at the end of the links properly.

- Use regular expressions to simplify parsing plain text.

- Strip spaces in link targets and text.

Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:45 -04:00
Sunil Mohan Adapa
3aca4cd164
doc: wikiparser: Allow empty lines between list items
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:43 -04:00
Sunil Mohan Adapa
9bb518cc76
doc: wikiparser: Parse content inside a comment
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:41 -04:00
Sunil Mohan Adapa
1f94ac4bce
doc: wikiparser: Handle more paragraph breakers
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:38 -04:00
Sunil Mohan Adapa
e5c51e9bb4
doc: wikiparser: Handle table of contents macro without parenthesis
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:36 -04:00
Sunil Mohan Adapa
64fedf1f32
doc: wikiparser: Generate colspec for tables
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:33 -04:00
Sunil Mohan Adapa
e947e99af3
doc: wikiparser: When processing single pages, ignore header/footer
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:30 -04:00
Sunil Mohan Adapa
4ef2244af4
doc: wikiparser: Fix attachment URLs in regular links
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:28 -04:00
Sunil Mohan Adapa
40f853278b
doc: wikiparser: Handle processing instructions
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-08-28 12:09:25 -04:00
James Valleroy
806db903cf
wikiparser: Fix spaces, multi-line, languages, icons
- Allow for additional languages to be added
- Fix unwanted spaces in paragraphs
- Handle multi-line ListItems
- Handle icons
- Add line break before inline image in list

Based on patches by Fioddor Superconcentrado <fiodor@gmail.com>.

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-08-28 12:09:05 -04:00
James Valleroy
5b0335a2fa
doc: Add moinmoin wiki parser
- Handle internal links
- Handle includes
- Handle generic categories and includes
- Handle embedded links
- Handle lists
- Handle admonitions and code blocks
- Generate docbook from wiki
- Use argparse
- Generate <itemizedlist> for lists
- Generate docbook article for testing
- Escape content strings
- Create nested <section>s for headings
- Handle links with empty text
- Generate comments as <remark> within <para>
- Add space at end of paragraph
- Ignore Category when generating
- Generate comment for horizontal rule
- Add missing structure to tables
- Don't generate End Include
- Handle Includes
- Handle BEGIN_INCLUDE
- Process begin and end markers when including file
- Handle embedded attachments
- Add a helper script to test all pages
- Generate docbook for admonitions
- Fix include context
- Don't generate for table of contents
- Ignore lines starting with ##
- Ignore space in comparison test
- Use <para> around listitem content
- Wrap code text in <screen>
- Only split plain text for double underscore
- Generate warning for smaller text
- Handle relative links to wiki
- Fix handling of recursive includes
- Parse URLs out of text
- Handle image attachment parameters
- Use SPDX license identifier
- Handle height property for images
- Strip whitespace when using startswith
- Add cols property to tgroup
- Handle spaces at start and end of multi-line CodeText
- Fix page names
- Make consistent in generating <para>s
- Rewrite list handling to support nested lists properly
- Handle embedded attachments inside links
- Handle links within emphasis
- Handle wiki links to InstallingDebianOn/
- Handle line breaks
- Use paragraphs in Admonitions.
- Handle Anchors
- Don't escape code text
- Fix stopping paragraph when list begins
- Fix characters dropped after PlainText
- Fix finding http and https links in text
- Ignore example URLs containing < or >
- Pass context to every call of to_docbook()
- Render CodeText in paragraph as simply <code>
- Ensure only tablestyle is removed from table cells
- Handle line breaks within table cells
- Handle text alignment style within table cell
- Only split plain text for double brace or double bracket
- Ignore lines with only whitespace for paragraphs
- Make tests verbose if no input files givena

Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
2020-08-28 12:08:58 -04:00
Fioddor Superconcentrado
f0000c4ba3
post-processor: Solve 1908 fixing the wiki links fix
The previous logic of the wiki links fix didn't work for translated pages.

Closes: #1908.

Signed-off-by: Fioddor Superconcentrado <fioddor@gmail.com>
[jvalleroy: Remove extra spaces]
[jvalleroy: Fix output path]
Signed-off-by: James Valleroy <jvalleroy@mailbox.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2020-07-25 10:12:27 -04:00
Sunil Mohan Adapa
07180c2309
*: Update misc build related files to use SPDX license identifier
These files are not installed on the system and don't end up in the Debian
package. They pose not threat of causing a configuration file prompt.

Reviewed-by: Veiko Aasa <veiko17@disroot.org>
2020-02-19 14:40:12 +02:00
Joseph Nuthalapati
d62463247f
doc: Move English manual to manual/en directory
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
[sunil@medhas.org Have common top level makefile for building manual]
Signed-off-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-10-30 19:25:02 -04:00
Joseph Nuthalapati
f475a94581
doc: Minor cosmetic changes
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-10-30 19:24:59 -04:00
Joseph Nuthalapati
004ba5b3ea
doc: Move build scripts into separate directory
Signed-off-by: Joseph Nuthalapati <njoseph@riseup.net>
Reviewed-by: Sunil Mohan Adapa <sunil@medhas.org>
Reviewed-by: James Valleroy <jvalleroy@mailbox.org>
2019-10-30 19:24:57 -04:00