Documentation not explicit on dependency on "libclang-dev", not having it results in "'NoneType' object has no attribute 'encode'"

[axsaucedo]

Documentation not explicit on dependency on "libcmake-dev", not having it results in "'NoneType' object has no attribute 'encode'".

Tested on master, on this file: https://github.com/EthicalML/vulkan-kompute/blob/master/single_include/kompute/Kompute.hpp

I also tried on a simpler file (https://github.com/EthicalML/vulkan-kompute/blob/master/src/include/kompute/Manager.hpp) but still get the same error:

Processing "Manager.hpp" ..
Waiting for jobs to finish ..
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/alejandro/miniconda3/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/alejandro/miniconda3/lib/python3.7/site-packages/pybind11_mkdoc/mkdoc_lib.py", line 232, in run
    tu = index.parse(self.filename, self.parameters)
  File "/home/alejandro/miniconda3/lib/python3.7/site-packages/clang/cindex.py", line 2722, in parse
    self)
  File "/home/alejandro/miniconda3/lib/python3.7/site-packages/clang/cindex.py", line 2816, in from_source
    args_array = (c_char_p * len(args))(*[b(x) for x in args])
  File "/home/alejandro/miniconda3/lib/python3.7/site-packages/clang/cindex.py", line 2816, in <listcomp>
    args_array = (c_char_p * len(args))(*[b(x) for x in args])
  File "/home/alejandro/miniconda3/lib/python3.7/site-packages/clang/cindex.py", line 109, in b
    return x.encode('utf8')
AttributeError: 'NoneType' object has no attribute 'encode'

Is this due to my environment? I'm testing in Ubuntu 18, Python3. Happy to provide further details.

[axsaucedo]

Ok I have just tested with the sample file and still get an error https://github.com/pybind/pybind11_mkdoc/blob/master/tests/sample_header_docs/sample_header.h.

The error is the same, is this something to do with my environment?

[axsaucedo]

Was able to replicate in docker image:

root@3f433f7e0613:/workspace# uname -a
Linux 3f433f7e0613 4.19.76-linuxkit #1 SMP Tue May 26 11:42:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

I also tested with branch v2.6 and seems to cause the same issue even with sample file.

It seems that this error tends to happen when particular libraries are not found numba/llvmlite#537, but a guess from the cmake python lib similar issue.

Are there any particular libraries that this depends on?

[axsaucedo]

Looking at the code it seems to fail when trying to find libclang, so installed libclang-dev, now I am facing the same issue as #2

[matthewhagraphcore]

The 'NoneType' object has no attribute 'encode' issue comes from these lines here. If you have an empty or incompletely installed llvm / clang then you get an None for the directory.

The stdlib.h issue I am yet to fix completely. Though I was able to fix this temporarily by changing the #include_next directives to #include in /usr/include/c++/9/cstdlib e.g. #include_next <stdlib.h> -> #include <stdlib.h>. Do at you own risk and keep backups!

[axsaucedo]

oh wow @matthewhagraphcore that worked... why?

[matthewhagraphcore]

I believe it occurs due to multiple gcc versions, some of which may be incorrectly installed. The #include_next directive will try and look for the next stdlib.h file, but can't find it (possibly it's looking for a newer version which isn't there, not sure).

Changing this to #include forces it to look in the same folder where we know the stdlib.h file is present.

Maybe removing all gcc installs and reinstalling might fix it, but I am on a shared server so was unwilling to try.

[axsaucedo]

@matthewhagraphcore thank you for the explanation. I see, that would make sense. I completely agree that requiring configuring the gcc setup is not really something reasonable even on a personal computer. Hopefully the issue that you raised results in the package being updated to support multiple versions of gcc. I will change the title of this issue to focus primarily on outlining that the "libcmake-dev" dependency not being made explicit in the setup seciton of the readme.

[henryiii]

Now I'm confused - why does libcmake-dev affect this? It's not using CMake at all. Is installing libcmake-dev triggering some other dependency to be installed forcing a valid dev environment, where otherwise parts were missing?

[axsaucedo]

Oh my bad, I meant libclang-dev not libcmake

pybind11_mkdoc/pybind11_mkdoc/mkdoc_lib.py

Line 268 in 0103839

library_file = sorted(glob("/usr/lib/llvm-*/lib/libclang.so.1"), reverse=True)[0]

[lujobi]

I just had the same problem using some std libraries (iostream among others). I found a solution which is by no means optimal but I was not comfortable at all editing /usr/include.

My idea is to add an extra Macro which is only defined when running pybind11_mkdoc. Thus clang doesn't try to import the library. I can live with the error error: use of undeclared identifier 'std' as long as the docstrings are created correctly.

#ifndef PYBIND11_MKDOC_EXTRACT_DOCSTRING
#include <iostream>
#endif

/**
 * Some docstring
 */
class ExampleClass {};

running with the following flag:

python3 -m pybind11_mkdoc -DPYBIND11_MKDOC_EXTRACT_DOCSTRING -o  ${DOCSTRING_FILE} ${FILES}

[leakec]

I'm going through old issues and trying to close out those that are no longer relevant or have already been fixed. I'm having trouble re-creating this on Ubuntu 18, since the most recent version of pybind11-mkdoc doesn't install on older Python versions.

I have re-done this in a clean docker image in Ubuntu 24

FROM ubuntu:24.04

# Update package lists and install sudo
RUN apt update
RUN apt install -y sudo python3-dev python3-pip git llvm clang
RUN python3 -m pip install --break-system-packages git+https://github.com/pybind/pybind11_mkdoc.git@master
RUN python3 -m pip install --break-system-packages clang==18.*

and everything works as expected. I ran this for the Manager.hpp file which in turn needed some other libraries libfmt-dev and libvuklan-dev, but those are of course project specific.

The only oddity in that docker image is installing the clang==18* python package. However, this is already being looked into here.

Please let me know if this is sufficient to close this issue. Thanks!

[leakec]

@henryiii Would it be okay to close this issue? This now works as intended with the current version, with the only hiccup being installing a specific version of clang (the Python package), but I believe there are already other places (#32) where alternatives are being looked into to avoid that hiccup.

[leakec]

@henryiii Just pinging again, as I think this can be closed.