Repository navigation
Add interface to override get_column_from_field | get_sqlalchemy_type function behavior #503
Description
Activity
- addedfeatureNew feature or requestNew feature or request
on Nov 20, 2022 I used this idea to extend / monkeypatch SQLModel to work with STRUCT and ARRAY from BigQuery:
# Monkeypatch sqlmodel.main.get_sqlalchemy_type to work with BigQuery's STRUCT and ARRAY types: from typing import Any, Callable, List import sqlalchemy.sql import sqlmodel.main from sqlmodel.main import get_sqlachemy_type as original_get_sqlalchemy_type from pydantic import BaseModel, ConstrainedStr from pydantic.fields import ModelField from sqlalchemy import String from sqlalchemy_bigquery import STRUCT, ARRAY from typing_extensions import TypeAlias _GetSqlalchemyTypeProtocol: TypeAlias = Callable[[ModelField], Any] def extended_get_sqlalchemy_type(field: ModelField) -> Any: # Add support for STRUCT if issubclass(field.type_, BaseModel): return STRUCT( *((f.name, extended_get_sqlalchemy_type(f)) for f in field.type_.__fields__.values()) ) # Add support for ARRAY if issubclass(field.type_, List): if field.sub_fields: return ARRAY(extended_get_sqlalchemy_type(field.sub_fields[0])) return ARRAY() # Add support for ConstrainedStr if issubclass(field.type_, ConstrainedStr): length = field.type_.max_length if length is not None: return String(length=length) return String() return original_get_sqlalchemy_type(field) def patch_sqlmodel() -> None: sqlmodel.main.get_sqlachemy_type = extended_get_sqlalchemy_type patch_sqlmodel()Now submodels can be used in SQLModel models and they get converted to STRUCT. e.g.:
class Service(SQLModel): id: str = Field(description="The ID of the Google Cloud Platform service that offers the SKU.") description: str = Field(description="The Google Cloud Platform service that offers the SKU.") class ServiceTable(SQLModel, table=True): service: Service = Field()I generally try to avoid this kind of patching shenanigans but this has greatly simplified my models and completely gotten rid of the need for
sa_columnin my SQLModel modelsReacted by 243f6a88 85a308d3I suggest to have a map that can be imported to modify and add type mappings between SQLModel and SQLAlchemy this also could solve #800
I have done something similar but to add additional support for different types my use case is to support the different JSON types of pydantic.
import sqlmodel.main from pydantic.types import Json as PydanticJson from pydantic import JsonValue as PydanticJsonValue from sqlalchemy.types import JSON # Exportable and modifiable TYPE_ANNOTATION_MAP = { PydanticJson: JSON, PydanticJsonValue: JSON } # Monkey patch for sqlmodel to support JSON type _gst = sqlmodel.main.get_sqlalchemy_type def get_sqlalchemy_type(field: Any) -> Any: try: return _gst(field) except Exception as exc: type_ = None if hasattr(field, 'type_'): type_ = field.type_ elif field.annotation: type_ = field.annotation if ( get_origin(type_) is Union and len(type_.__args__) == 2 and type(None) in type_.__args__ ): type_ = next(arg for arg in type_.__args__ if arg is not None) if get_origin(type_) is Annotated: type_ = type(type_.__metadata__[0]) if type_ in TYPE_ANNOTATION_MAP.keys(): return TYPE_ANNOTATION_MAP[type_] raise exc sqlmodel.main.get_sqlalchemy_type = get_sqlalchemy_type
The functionality in the first comment of this issue can be added by accept
Callablesalso with SQLAlchemy types, for examplefrom sqlmodel.main import TYPE_ANNOTATION_MAP TYPE_ANNOTATION_MAP.extend({ PydanticJson: JSON, PydanticJsonValue: JSON, ConstrainedStr: lambda f: String(length=field.type_.max_length) })
If you want I could do a PR with this.
Cheers
PS: I have to mention and thanks to ChatLPO that allows me to post this suggestions based on the work in the company.
- locked and limited conversation to collaborators
on May 18, 2026
First Check
Commit to Help
Example Code
Description
Problem:
SQLModeldoes not provide expected column type, and it is (in some cases) impossible because requirements ofSQLModelusers are not always the same (e.g. DBMS dialects, strictness of constraints, choice of CHAR vs VARCHAR vs TEXT vs JSON, TIMESTAMP vs DATETIME)Wanted Solution
Allow user to use customized
get_column_from_field|get_sqlalchemy_typefunction to fit with their own requirements.Add parameter to model config like
sa_column_builder: Callable[[ModelField], Column] = get_column_from_field.Function
get_column_from_fieldwould be better split by the following concerns, to be used as a part of customizedsa_column_builderimplementation:get_sqlalchemy_type)Possible effects on other issues/PRs:
sqlmodelusers:p.s-1
Conversion rule between Field/column value may become necessary, mainly to serialize field value to column value.
(e.g. Classes inheriting BaseModel cannot be stored directly into
sqlalchemy.JSONbecause it is not JSON or dict. We avoid this by addingjson_serializertocreate_engine. Deserialize part has no problem because JSONstr->BaseModelwill be done by pydantic validation for now (pydantic v1))p.s-2
IMO using
sqlmodel.sql.sqltypes.AutoString()in alembic revision file is not good from the sight of future migration constancy, and this is one of the reason I overriddenget_sqlalchemy_typefunction.Wanted Code
Alternatives
sa_columnand call it insqlmodel.FielddeclarationConstrainedStr, ...)Operating System
Linux
Operating System Details
No response
SQLModel Version
0.0.8
Python Version
3.10.7
Additional Context
No response