-
Notifications
You must be signed in to change notification settings - Fork 56
update multimodal part #7
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements comprehensive multimodal file support for the chat application, allowing users to upload and send images, audio, video, and documents alongside text messages. The changes span frontend file handling, backend processing, and type definitions.
Key changes:
- Added file attachment functionality with drag-and-drop, paste, and file picker support
- Implemented multimodal message processing with base64 encoding for smaller files and Gemini file upload for larger files
- Updated type definitions and UI components to display uploaded files in conversations
Reviewed Changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 27 comments.
Show a summary per file
File | Description |
---|---|
frontend/app/stores/sessionStore.ts | Added file processing and base64 conversion to message sending |
frontend/app/chat/types/conversation.ts | Extended Turn interface to include files array in inputs |
frontend/app/chat/page.tsx | Updated sendMessage to accept optional files parameter |
frontend/app/chat/components/details/TurnBubble.tsx | Added display logic for uploaded files in chat bubbles |
frontend/app/chat/components/WelcomeScreen.tsx | Replaced custom input with ChatInput component |
frontend/app/chat/components/ProjectPage.tsx | Replaced custom input with ChatInput component |
frontend/app/chat/components/ChatInput.tsx | Complete rewrite with multimodal file support |
core/pyproject.toml | Added httpx dependency |
core/api/message_handlers.py | Enhanced message handling for multimodal payloads |
core/agent_core/nodes/mcp_proxy_node.py | Added path normalization for Windows compatibility |
core/agent_core/nodes/base_agent_node.py | Added multimodal content preservation and message validation |
core/agent_core/framework/inbox_processor.py | Implemented comprehensive file processing with Gemini integration |
core/agent_core/events/ingestors.py | Added multimodal ingestor for file attachments |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
No description provided.