Skip to content

feat: add GenUI support for Omi chat (#3810)#6816

Open
tejasprasad2008-afk wants to merge 3 commits intoBasedHardware:mainfrom
tejasprasad2008-afk:feat/genui-omi-chat
Open

feat: add GenUI support for Omi chat (#3810)#6816
tejasprasad2008-afk wants to merge 3 commits intoBasedHardware:mainfrom
tejasprasad2008-afk:feat/genui-omi-chat

Conversation

@tejasprasad2008-afk
Copy link
Copy Markdown

  • Add GenUIMessageWidget for location_prompt and map_result components
  • Update markdown_message_widget to detect and render GenUI JSON
  • Pass sendMessage callback through NormalMessageWidget and MemoriesMessageWidget
  • Add GenUI instructions to backend system prompt in chat.py
  • Backward compatible: plain text messages unaffected

- Add GenUIMessageWidget for location_prompt and map_result components
- Update markdown_message_widget to detect and render GenUI JSON
- Pass sendMessage callback through NormalMessageWidget and MemoriesMessageWidget
- Add GenUI instructions to backend system prompt in chat.py
- Backward compatible: plain text messages unaffected
@tejasprasad2008-afk
Copy link
Copy Markdown
Author

Closes #3810

What this does

Replaces markdown rendering with interactive GenUI components in Omi chat.

Changes

  • backend/utils/llm/chat.py — system prompt instructs AI to return GenUI JSON for location queries
  • genui_message_widget.dart — new widget: renders location_prompt (Yes/No + geolocator) and map_result (FlutterMap pin)
  • markdown_message_widget.dart — detects GenUI JSON before falling back to markdown
  • ai_message.dart — sendMessage callback plumbed through to enable button interactions

Flow

  1. User asks "where is the nearest X?" → AI returns raw JSON
  2. Flutter detects JSON → renders interactive card instead of text
  3. User taps Yes → GPS coordinates sent back to AI
  4. AI returns map_result JSON → FlutterMap renders with pin

Backward compatible

All non-genui messages continue rendering as markdown exactly as before.

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 18, 2026

Greptile Summary

This PR adds GenUI support to the Omi chat, enabling the LLM to return structured JSON that renders interactive Flutter widgets (location prompts, map cards, result cards) instead of plain markdown. The backend injects GenUI instructions into the RAG prompt, and the Flutter side detects and dispatches the JSON to the appropriate widget.

  • P0 — Compile failure: markdown_message_widget.dart imports genui_widget.dart (file does not exist) and references GenUIMessageWidget (class never defined); the app will not build until both are removed — the location_prompt branch is fully redundant with the existing _buildGenUIWidget path.
  • P1 — Malformed prompt tag: _get_qa_rag_prompt uses <question> instead of </question> as the closing tag, leaving the question content and memories block in a single unclosed element.
  • P1 — GenUI instructions missing from non-RAG paths: _get_answer_simple_message_prompt and the agentic prompt never receive the GenUI instructions, so map/location cards will never appear for questions that don't trigger retrieval.

Confidence Score: 3/5

Not safe to merge — the app will not compile due to a missing file import and undefined class reference

A P0 compile error (missing genui_widget.dart import + undefined GenUIMessageWidget) means the PR cannot ship as-is. Two additional P1 issues (malformed XML tag in prompt, GenUI instructions absent from non-RAG chat paths) would cause silent feature gaps even after the build is fixed.

markdown_message_widget.dart (compile error), backend/utils/llm/chat.py (malformed tag + missing GenUI instructions in simple/agentic paths)

Important Files Changed

Filename Overview
app/lib/pages/chat/widgets/markdown_message_widget.dart Rewired to detect and render GenUI JSON; imports non-existent genui_widget.dart and uses undefined GenUIMessageWidget, causing a compile failure
app/lib/pages/chat/widgets/genui_message_widget.dart New file adding GenUI data model and widgets (LocationSharePrompt, MapCard, ResultCard, ConfirmPrompt); hardcodes iOS package name in OSM tile user-agent on both map widgets
app/lib/pages/chat/widgets/ai_message.dart Threads sendMessage callback through NormalMessageWidget and MemoriesMessageWidget; one child: line lost its indentation
backend/utils/llm/chat.py Adds GenUI instructions to _get_qa_rag_prompt only (missing from simple and agentic paths) and has a malformed <question> closing XML tag; rest of changes are quote-style normalization

Sequence Diagram

sequenceDiagram
    participant User
    participant Flutter as Flutter Chat UI
    participant Backend as Backend (chat.py)
    participant LLM

    User->>Flutter: sends message
    Flutter->>Backend: POST /chat message
    Backend->>LLM: prompt with genui_instructions (RAG path only)
    LLM-->>Backend: JSON genui location_prompt
    Backend-->>Flutter: message text (raw JSON string)
    Flutter->>Flutter: getMarkdownWidget()
    Flutter->>Flutter: GenUIMessage.tryParse()
    alt component == location_prompt
        Flutter->>Flutter: render LocationSharePrompt widget
        User->>Flutter: taps Yes
        Flutter->>Backend: sendMessage with coordinates
        Backend->>LLM: follow-up with lat/lng
        LLM-->>Backend: JSON map_result
        Backend-->>Flutter: map_result JSON
        Flutter->>Flutter: render MapCard widget
    else other component
        Flutter->>Flutter: render appropriate GenUI widget
    end
Loading

Comments Outside Diff (1)

  1. backend/utils/llm/chat.py, line 392-394 (link)

    P1 Malformed XML closing tag may confuse the LLM

    The <question> block is never closed — line 394 repeats the opening tag instead of using </question>. LLMs parse prompt structure literally, so this leaves the question content and <memories> block inside what appears to be a single unclosed element, which may degrade response quality.

Reviews (1): Last reviewed commit: "feat: add GenUI support for Omi chat (#3..." | Re-trigger Greptile

Comment thread app/lib/pages/chat/widgets/markdown_message_widget.dart Outdated
Comment thread app/lib/pages/chat/widgets/markdown_message_widget.dart Outdated
Comment thread backend/utils/llm/chat.py
Comment thread app/lib/pages/chat/widgets/ai_message.dart Outdated
Comment thread app/lib/pages/chat/widgets/genui_message_widget.dart
@tejasprasad2008-afk
Copy link
Copy Markdown
Author

All Greptile review issues have been addressed in the latest commit (15ca7a1):

  • P0 fixed: Removed dead `genui_widget.dart` import and redundant `GenUIMessageWidget` reference from `markdown_message_widget.dart` — app now compiles cleanly
  • P1 fixed: Added GenUI instructions to both `_get_answer_simple_message_prompt` and `_get_agentic_qa_prompt` paths in `chat.py` — not just the RAG path
  • P1 fixed: Corrected malformed `` closing tag in `chat.py`
  • P2 fixed: Restored proper indentation on `child:` line in `ai_message.dart`
  • P2 fixed: Replaced hardcoded `com.friend.ios` user-agent with `com.based.omi` in both map widgets

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant