Ahmed Galadima
56d56395ee
- Added "audioCapture" permission to manifest for microphone access. - Introduced DeepSeek as a new AI provider option in the side panel. - Implemented a capture mode selection (tab-only, mic-only, mixed) in the side panel. - Added options to enable/disable the extension and auto-open the assistant window. - Integrated a mic monitor feature with live input level visualization. - Included buttons for requesting microphone permission and granting tab access. - Updated styles for new sections and mic level visualization. - Enhanced model fetching logic to support DeepSeek and improved error handling.
2.5 KiB
2.5 KiB
Personal Browser Companion - Plans & To-Do
Goals
- Start local-first with an option to sync to cloud.
- Online-only operation (LLM required for decisions).
- Auto-start mode during meetings.
- Integrations: calendar, email, Discord, Nextcloud.
Phase Plan
Phase 1: Local MVP (Foundation)
- Local storage for sessions, summaries, and user profile.
- Meeting/interview modes with manual start and overlay UI.
- Basic memory retrieval: recent session summaries + user profile.
- Audio capture + STT pipeline (mic + tab) and transcript display.
- Privacy controls: store/forget, per-session toggle.
Phase 2: Smart Auto-Start
- Detect meeting tabs (Google Meet, Zoom, Teams) and prompt to start.
- Auto-start rules (domain allowlist, time-based, calendar hints).
- Lightweight on-device heuristics for meeting detection.
Phase 3: Cloud Sync (Optional)
- Opt-in cloud sync for memory + settings.
- Conflict resolution strategy (last-write wins + merge for summaries).
- Encryption at rest, user-controlled delete/export.
Phase 4: Integrations (MCP)
- Calendar: read upcoming meetings, attach context.
- Email: draft follow-ups, summaries.
- Discord: post meeting summary or action items to a channel.
- Nextcloud: store meeting notes, transcripts, and attachments.
MVP To-Do (Local)
Core
- Define memory schema (profile, session, summary, action items).
- Implement local RAG: index summaries + profile into embeddings.
- Add session lifecycle: start, pause, end, summarize.
Audio + STT
- Implement reliable STT for tab audio (server-side if needed).
- Keep mic-only STT as fallback.
- Add device selection + live mic monitor.
UI/UX
- Overlay controls: resize, hide/show, minimize.
- Auto-start toggle in side panel.
- Session summary view with “save to memory” toggle.
Privacy
- Per-session storage consent prompt.
- “Forget session” button.
Integration To-Do (MCP)
MCP Server Options
- Build a local MCP server as a bridge for integrations.
- Use MCP tool registry for calendar/email/Discord/Nextcloud.
Calendar
- Read upcoming meetings and titles.
- Auto-attach relevant context packs.
- Generate follow-up drafts from summary + action items.
Discord
- Post meeting summary/action items to a selected channel.
Nextcloud
- Upload meeting notes and transcripts.
Open Questions
- Preferred cloud provider for sync?
- How long should session memories persist by default?
- Should auto-start be opt-in per domain or global?
- What data should be redacted before sync?