Skip to content

Plugin output is not being rendered #702

Description

@jessicah

Describe the bug

context
I've written a port of markdown-it-abbr to python, but when used with MyST-Parser, the replacement tokens are missing, due to an error of WARNING: No render method for: abbr [myst.render].

expectation
In the rendered HTML, I expected the <abbr> HTML tag and content to be output.

E.g. the following markdown:

*[HTML]: HyperText Markup Language
*[W3C]: World Wide Web Consortium

The HTML specification is maintained by the W3C.

should generate:

<p>The <abbr title="HyperText Markup Language'>HTML</abbr> specification is maintained by the <abbr title="World Wide Web Consortium">W3C</abbr>.</p>

bug
Instead, it generates:

<p>The specification is maintained by the .</p>

I added print(nodes) at https://github.com/jessicah/mdit-py-abbr/blob/main/mdit_py_abbr/index.py#L157, and it showed up fine in console:

[Token(type='text', tag='', nesting=0, attrs={}, map=None, level=0, children=None, content='The ', markup='', info='', meta={}, block=False, hidden=False), Token(type='abbr_open', tag='abbr', nesting=1, attrs={'title': 'HyperText Markup Language'}, map=None, level=0, children=None, content='', markup='', info='', meta={}, block=False, hidden=False), Token(type='text', tag='', nesting=0, attrs={}, map=None, level=0, children=None, content='HTML', markup='', info='', meta={}, block=False, hidden=False), Token(type='abbr_close', tag='abbr', nesting=-1, attrs={}, map=None, level=0, children=None, content='', markup='', info='', meta={}, block=False, hidden=False), Token(type='text', tag='', nesting=0, attrs={}, map=None, level=0, children=None, content=' specification is maintained by the ', markup='', info='', meta={}, block=False, hidden=False), Token(type='abbr_open', tag='abbr', nesting=1, attrs={'title': 'World Wide Web Consortium'}, map=None, level=0, children=None, content='', markup='', info='', meta={}, block=False, hidden=False), Token(type='text', tag='', nesting=0, attrs={}, map=None, level=0, children=None, content='W3C', markup='', info='', meta={}, block=False, hidden=False), Token(type='abbr_close', tag='abbr', nesting=-1, attrs={}, map=None, level=0, children=None, content='', markup='', info='', meta={}, block=False, hidden=False), Token(type='text', tag='', nesting=0, attrs={}, map=None, level=0, children=None, content='.', markup='', info='', meta={}, block=False, hidden=False)]

So the token stream itself seems correct. Also, using markdown-it-py directly works correctly.

I did try adding a render method, but it had no effect. E.g.:

md.add_render_rule("abbr", abbr_render)

and

def abbr_render(self, tokens, idx, options, env):
    return f'<abbr title="{tokens[idx].attrGet("title")}">{tokens[idx].content}</abbr>'

Reproduce the bug

I've created an example repo: https://github.com/jessicah/myst-demo

make [all] runs sphinx-build, and make test runs the test file.

List your environment

myst_parser: 0.18.1
sphinx: 5.3.0
docutils: 0.17.1
markdown_it: 1.1.0

Activity

  1. welcome commented on Feb 17, 2023

    @welcome

    Thanks for opening your first issue here! Engagement like this is essential for open source projects! 🤗

    If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.

    If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).

    Welcome to the EBP community! 🎉

  2. jessicah commented on Feb 17, 2023

    @jessicah
    Author

    It seems like perhaps MyST-Parser's DocutilsRenderer is the culprit, using a fixed list of renderers?

    See: https://github.com/executablebooks/MyST-Parser/blob/master/myst_parser/mdit_to_docutils/base.py#L102-L106

    This might also explain what was going wrong in my WIP patch in #632

  3. jessicah commented on Feb 17, 2023

    @jessicah
    Author

    Ohh, I see, it's because the overridden render methods produces docutils AST nodes, correct? And the plugins produce HTML strings, which is why all the currently supported plugins have their equivalent render_xyz methods in the DocutilsRenderer class. Still, an easy way to extend them would be useful... hmm.

  4. chrisjsewell commented on Feb 17, 2023

    @chrisjsewell
    Member

    Heya, yep indeed myst-parser intentionally does not expose markdown-it-py as the underlying markdown parser, it would not be maintainable to support such an API.

    If you want to have a look at contributing your plugin to https://github.com/executablebooks/mdit-py-plugins, we can look to add it to myst-parser as an extension 😄
    (there is already a node type for it, so should not be too difficult https://docutils.sourceforge.io/docs/ref/doctree.html#abbreviation)

  5. jessicah commented on Feb 17, 2023

    @jessicah
    Author

    @chrisjsewell ah, yeah, I was just at that docutils link, although not quite sure how to write the render_abbr method in my python code.

    I figured I can make a custom renderer by subclassing myst_parser.mdit_to_docutils.sphinx_.SphinxRenderer, and calling create_md_parser(config, CustomRenderer):

    class CustomRenderer(SphinxRenderer):
    	def render_abbr(self, token: SyntaxTreeNode) -> None:
    		print(token)
    
    class CustomParser(MystParser):
    	def parse(self, inputstring: str, document: nodes.document) -> None:
    		config: MdParserConfig = document.settings.env.myst_config
    		# . . .			
    		parser = create_md_parser(config, CustomRenderer)
    		# . . . 
    
    def setup(app):
    	from myst_parser.sphinx_ext.main import setup_sphinx
    
    	setup_sphinx(app, load_parser=False)
    	app.add_source_suffix(".md", "markdown")
    	app.add_source_parser(CustomParser)
    

    I'm at least getting the print outputing SyntaxTreeNode(abbr) in my test, so it's just figuring out the final piece...

    Some tips on what I should be producing in render_abbr would be great, and I can make PRs for both :)

  6. chrisjsewell commented on Feb 17, 2023

    @chrisjsewell
    Member

    Yeh no worries, if you get the PR into executablebooks/mdit-py-plugins, I should be able to take care of the myst-parser side (it should be relatively straight forward)

  7. jessicah commented on Feb 17, 2023

    @jessicah
    Author
  8. jessicah commented on Feb 18, 2023

    @jessicah
    Author

    @chrisjsewell any chance you can help me out with the myst-parser side as the plugin exists atm? I'd like to be able to start using in my own project ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions