Griffe fail to work with Pydantic models constrained types

[dmulyalin]

Describe the bug

I am using Pydantic in my project with contstrained types, Griffe seems to fail working with them.

To Reproduce

Install Pydantic - pip install pydantic

1. Create directory foo and file foo/sample_pydantic_model.py with this content:

from pydantic import BaseModel, conint
from typing import Optional

class Foo(BaseModel):
    # enforce count greater then (gt) 0 constraint
    count: Optional[conint(strict=True, gt=0)]

2. Run command

griffe foo -L DEBUG

to get this error:

(docsbuilder) ..some_path\>griffe foo -L DEBUG
INFO       Loading package foo
DEBUG      Found foo: loading
DEBUG      Loading path [WindowsPath('foo'), WindowsPath('...some_path/foo')]
DEBUG      Loading path foo\sample_pydantic_model.py
Traceback (most recent call last):
  File "C:\Users\someuser\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\someuser\AppData\Local\Programs\Python\Python37-32\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "...some_path\venv\docsbuilder\Scripts\griffe.exe\__main__.py", line 7, in <module>
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\cli.py", line 311, in main
    not opts.no_inspection,
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\cli.py", line 128, in _load_packages
    loader.load_module(package)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 148, in load_module
    top_module = self._load_module(package.name, package.path, submodules=submodules)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 346, in _load_module
    return self._load_module_path(module_name, module_path, submodules, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 374, in _load_module_path
    self._load_submodules(module)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 379, in _load_submodules
    self._load_submodule(module, subparts, subpath)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 391, in _load_submodule
    subparts[-1], subpath, submodules=False, parent=member_parent
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 346, in _load_module
    return self._load_module_path(module_name, module_path, submodules, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 368, in _load_module_path
    module = self._visit_module(code, module_name, module_path, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\loader.py", line 416, in _visit_module
    modules_collection=self.modules_collection,
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 101, in visit
    modules_collection=modules_collection,
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 173, in get_module
    self.visit(top_node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 184, in visit
    super().visit(node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\base.py", line 19, in visit
    getattr(self, f"visit_{node.kind}", self.generic_visit)(node)  # type: ignore[attr-defined]
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 215, in visit_module
    self.generic_visit(node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 197, in generic_visit
    self.visit(child)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 184, in visit
    super().visit(node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\base.py", line 19, in visit
    getattr(self, f"visit_{node.kind}", self.generic_visit)(node)  # type: ignore[attr-defined]
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 255, in visit_classdef
    self.generic_visit(node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 197, in generic_visit
    self.visit(child)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 184, in visit
    super().visit(node)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\base.py", line 19, in visit
    getattr(self, f"visit_{node.kind}", self.generic_visit)(node)  # type: ignore[attr-defined]
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\visitor.py", line 602, in visit_annassign
    self.handle_attribute(node, get_annotation(node.annotation, parent=self.current))
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 741, in get_annotation
    return _get_annotation(node, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 726, in _get_annotation
    return _node_annotation_map[type(node)](node, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 675, in _get_subscript_annotation
    subscript = _get_annotation(node.slice, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 726, in _get_annotation
    return _node_annotation_map[type(node)](node, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 708, in _get_index_annotation
    return _get_annotation(node.value, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 726, in _get_annotation
    return _node_annotation_map[type(node)](node, parent)
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 630, in _get_call_annotation
    kwargs = Expression(*[_get_annotation(kwarg, parent) for kwarg in node.keywords])
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 630, in <listcomp>
    kwargs = Expression(*[_get_annotation(kwarg, parent) for kwarg in node.keywords])
  File "...some_path\venv\docsbuilder\lib\site-packages\griffe\agents\nodes.py", line 726, in _get_annotation
    return _node_annotation_map[type(node)](node, parent)
KeyError: <class '_ast.keyword'>

Expected behavior

Should not get above error

System (please complete the following information):

• OS: Windows

venv version report:

(docsbuilder) ..some_path\>python3 -m pip list
Package                    Version
-------------------------- -------
astunparse                 1.6.3
bcrypt                     3.2.0
cached-property            1.5.2
cffi                       1.15.0
click                      8.1.2
colorama                   0.4.4
cryptography               36.0.2
fakenos                    0.1.0
ghp-import                 2.0.2
griffe                     0.18.0
importlib-metadata         4.11.3
Jinja2                     3.1.1
Markdown                   3.3.6
MarkupSafe                 2.1.1
mergedeep                  1.3.4
mkdocs                     1.3.0
mkdocs-autorefs            0.4.1
mkdocs-material            8.2.9
mkdocs-material-extensions 1.0.3
mkdocstrings               0.18.1
mkdocstrings-python        0.6.6
mkdocstrings-python-legacy 0.2.2
packaging                  21.3
paramiko                   2.10.3
pip                        22.0.4
pycparser                  2.21
pydantic                   1.9.0
Pygments                   2.11.2
pymdown-extensions         9.3
PyNaCl                     1.5.0
pyparsing                  3.0.8
python-dateutil            2.8.2
pytkdocs                   0.16.1
PyYAML                     6.0
pyyaml_env_tag             0.1
setuptools                 39.0.1
six                        1.16.0
typing_extensions          4.2.0
watchdog                   2.1.7
wheel                      0.37.1
zipp                       3.8.0

Additional context

Griffe runs fine with any other pydantic models and only fails as soon as I start using constrained types, e.g. this code works ok with Griffe:

from pydantic import BaseModel, conint
from typing import Optional

class Foo(BaseModel):
    # enforce count greater then (gt) 0 constraint
    count: Optional[int]

[pawamoy]

Hello, thanks for the report!

Indeed I didn't implement support for function calls when getting annotations yet. It would be easy to do though 👍
But I wonder: with such an annotation Optional[conint(strict=True, gt=0)], what would you like to see in your docs? The same thing, i.e. Optional[conint(strict=True, gt=0)], or something else?

[dmulyalin]

To be honest, personally have no plans to use Griffe to pull docstrings out of Pydantic models, it rather that Griffe scans all the files in my project when it collecting whatever its collecting, it just happens that some of these files contains aforementioned pydantic models with constrained types.

But, IMHO, conint theoretically should evaluate to int in typing annotations, conlist to list, constr to str etc.

[pawamoy]

I see. It's a design decision I made when writing Griffe: it would load everything and be fast enough to do so.

[pawamoy]

This should be fixed. Will release soon.

[dmulyalin]

@pawamoy Thank you for prompt response on this one. Appreciate your time and efforts.

[pawamoy]

It has taken a bit more time than I thought, but 0.19.0 is out 🙂
Let me know if there's any remaining issue!