Ahmed Galadima
56d56395ee
- Added "audioCapture" permission to manifest for microphone access. - Introduced DeepSeek as a new AI provider option in the side panel. - Implemented a capture mode selection (tab-only, mic-only, mixed) in the side panel. - Added options to enable/disable the extension and auto-open the assistant window. - Integrated a mic monitor feature with live input level visualization. - Included buttons for requesting microphone permission and granting tab access. - Updated styles for new sections and mic level visualization. - Enhanced model fetching logic to support DeepSeek and improved error handling.
79 lines
2.5 KiB
Markdown
79 lines
2.5 KiB
Markdown
# Personal Browser Companion - Plans & To-Do
|
|
|
|
## Goals
|
|
- Start local-first with an option to sync to cloud.
|
|
- Online-only operation (LLM required for decisions).
|
|
- Auto-start mode during meetings.
|
|
- Integrations: calendar, email, Discord, Nextcloud.
|
|
|
|
## Phase Plan
|
|
|
|
### Phase 1: Local MVP (Foundation)
|
|
- Local storage for sessions, summaries, and user profile.
|
|
- Meeting/interview modes with manual start and overlay UI.
|
|
- Basic memory retrieval: recent session summaries + user profile.
|
|
- Audio capture + STT pipeline (mic + tab) and transcript display.
|
|
- Privacy controls: store/forget, per-session toggle.
|
|
|
|
### Phase 2: Smart Auto-Start
|
|
- Detect meeting tabs (Google Meet, Zoom, Teams) and prompt to start.
|
|
- Auto-start rules (domain allowlist, time-based, calendar hints).
|
|
- Lightweight on-device heuristics for meeting detection.
|
|
|
|
### Phase 3: Cloud Sync (Optional)
|
|
- Opt-in cloud sync for memory + settings.
|
|
- Conflict resolution strategy (last-write wins + merge for summaries).
|
|
- Encryption at rest, user-controlled delete/export.
|
|
|
|
### Phase 4: Integrations (MCP)
|
|
- Calendar: read upcoming meetings, attach context.
|
|
- Email: draft follow-ups, summaries.
|
|
- Discord: post meeting summary or action items to a channel.
|
|
- Nextcloud: store meeting notes, transcripts, and attachments.
|
|
|
|
## MVP To-Do (Local)
|
|
|
|
### Core
|
|
- Define memory schema (profile, session, summary, action items).
|
|
- Implement local RAG: index summaries + profile into embeddings.
|
|
- Add session lifecycle: start, pause, end, summarize.
|
|
|
|
### Audio + STT
|
|
- Implement reliable STT for tab audio (server-side if needed).
|
|
- Keep mic-only STT as fallback.
|
|
- Add device selection + live mic monitor.
|
|
|
|
### UI/UX
|
|
- Overlay controls: resize, hide/show, minimize.
|
|
- Auto-start toggle in side panel.
|
|
- Session summary view with “save to memory” toggle.
|
|
|
|
### Privacy
|
|
- Per-session storage consent prompt.
|
|
- “Forget session” button.
|
|
|
|
## Integration To-Do (MCP)
|
|
|
|
### MCP Server Options
|
|
- Build a local MCP server as a bridge for integrations.
|
|
- Use MCP tool registry for calendar/email/Discord/Nextcloud.
|
|
|
|
### Calendar
|
|
- Read upcoming meetings and titles.
|
|
- Auto-attach relevant context packs.
|
|
|
|
### Email
|
|
- Generate follow-up drafts from summary + action items.
|
|
|
|
### Discord
|
|
- Post meeting summary/action items to a selected channel.
|
|
|
|
### Nextcloud
|
|
- Upload meeting notes and transcripts.
|
|
|
|
## Open Questions
|
|
- Preferred cloud provider for sync?
|
|
- How long should session memories persist by default?
|
|
- Should auto-start be opt-in per domain or global?
|
|
- What data should be redacted before sync?
|