112 lines
5.4 KiB
Markdown
112 lines
5.4 KiB
Markdown
# Personal Browser Companion - Plans & To-Do
|
|
|
|
## Classification
|
|
- Core = works in extension-only mode (no extra server required).
|
|
- Optional = requires extra server/services (MCP, cloud sync, external APIs) and is opt-in per user.
|
|
|
|
## Goals
|
|
- [ ] [Core] Start local-first with an option to sync to cloud.
|
|
- [ ] [Core] Online-only operation (LLM required for decisions).
|
|
- [ ] [Core] Auto-start mode during meetings.
|
|
- [ ] [Optional] Integrations: calendar, email, Discord, Nextcloud.
|
|
|
|
## Phase Plan
|
|
|
|
### Phase 1: Local MVP (Foundation) [Core]
|
|
- [x] [Core] Local storage for sessions, summaries, and user profile.
|
|
- [x] [Core] Meeting/interview modes with manual start and overlay UI.
|
|
- [x] [Core] Basic memory retrieval: recent session summaries + user profile.
|
|
- [ ] [Core] Audio capture + STT pipeline (mic + tab) and transcript display.
|
|
- [x] [Core] Privacy controls: store/forget, per-session toggle.
|
|
|
|
### Phase 2: Smart Auto-Start [Core]
|
|
- [ ] [Core] Detect meeting tabs (Google Meet, Zoom, Teams) and prompt to start.
|
|
- [ ] [Core] Auto-start rules (domain allowlist, time-based, calendar hints).
|
|
- [ ] [Core] Lightweight on-device heuristics for meeting detection.
|
|
|
|
### Phase 3: Cloud Sync (Optional) [Optional]
|
|
- [ ] [Optional] Opt-in cloud sync for memory + settings.
|
|
- [ ] [Optional] Conflict resolution strategy (last-write wins + merge for summaries).
|
|
- [ ] [Optional] Encryption at rest, user-controlled delete/export.
|
|
|
|
### Phase 4: Integrations (MCP) [Optional]
|
|
- [ ] [Optional] Calendar: read upcoming meetings, attach context.
|
|
- [ ] [Optional] Email: draft follow-ups, summaries.
|
|
- [ ] [Optional] Discord: post meeting summary or action items to a channel.
|
|
- [ ] [Optional] Nextcloud: store meeting notes, transcripts, and attachments.
|
|
|
|
## MVP To-Do (Local)
|
|
|
|
### Core
|
|
- [x] [Core] Define memory schema (profile, session, summary, action items).
|
|
- [x] [Core] Implement local RAG: index summaries + profile into embeddings.
|
|
- [x] [Core] Add session lifecycle: start, pause, end, summarize.
|
|
|
|
### Audio + STT
|
|
- [x] [Core] Implement reliable STT for tab audio (OpenAI Whisper chunk transcription from tab/mixed audio).
|
|
- [x] [Core] Keep mic-only STT as fallback.
|
|
- [x] [Core] Add device selection + live mic monitor.
|
|
- [x] [Core] Add separate STT settings (provider/model/API key) independent from chat provider.
|
|
- [x] [Optional] Add local STT bridge support (self-hosted faster-whisper endpoint).
|
|
- [x] [Core] Add STT "Test Connection" action in Assistant Setup.
|
|
- [x] [Core] Add multilingual STT controls (auto/forced language, task, VAD, beam size) with session language lock in auto mode.
|
|
|
|
### UI/UX
|
|
- [x] [Core] Overlay controls: resize, hide/show, minimize.
|
|
- [x] [Core] Auto-start toggle in side panel.
|
|
- [x] [Core] Session summary view with “save to memory” toggle.
|
|
- [x] [Core] Sidebar automation preset selector before Start Listening.
|
|
- [x] [Core] One-click session context selector before Start Listening.
|
|
- [x] [Core] Profile-scoped context loading to reduce cross-session prompt leakage.
|
|
- [x] [Core] Profile manager UI (create/edit/delete profile with mode + prompt).
|
|
- [ ] [Core] Import/export context profiles.
|
|
|
|
### Privacy
|
|
- [x] [Core] Per-session storage consent prompt.
|
|
- [x] [Core] “Forget session” button.
|
|
|
|
### Advanced Settings (Core)
|
|
- [x] [Core] Open full settings window from side panel (⚙️).
|
|
- [x] [Core] Webhook test: send sample payload and show status.
|
|
- [x] [Core] MCP connection test (basic reachability).
|
|
- [x] [Core] Cloud endpoint validation (basic reachability).
|
|
- [ ] [Core] Automation framework: triggers + actions + approval flow.
|
|
|
|
## Integration To-Do (MCP)
|
|
|
|
### MCP Server Options
|
|
- [ ] [Optional] Build a local MCP server as a bridge for integrations.
|
|
- [ ] [Optional] Use MCP tool registry for calendar/email/Discord/Nextcloud.
|
|
|
|
### Automation (Rules Engine)
|
|
- [ ] [Core] Configure triggers (session start/end/manual, meeting domain filters).
|
|
- [ ] [Core] Configure actions per trigger (MCP tool + args).
|
|
- [ ] [Core] Approval mode: auto-send or review before send.
|
|
- [ ] [Core] Run actions on session end (hook into session lifecycle).
|
|
- [ ] [Core] Manual “Run Actions” button.
|
|
|
|
### Calendar
|
|
- [ ] [Optional] Read upcoming meetings and titles.
|
|
- [ ] [Optional] Auto-attach relevant context packs.
|
|
|
|
### Email
|
|
- [ ] [Optional] Generate follow-up drafts from summary + action items.
|
|
|
|
### Discord
|
|
- [ ] [Optional] Post meeting summary/action items to a selected channel.
|
|
|
|
### Nextcloud
|
|
- [ ] [Optional] Upload meeting notes and transcripts.
|
|
|
|
## Open Questions
|
|
- [Core] How do we isolate interview vs meeting prompts/contexts safely?
|
|
- Best solution: Use explicit context profiles (e.g., Interview, Standup, Sales) with separate prompt + context store per profile, and require users to pick one profile before Start Listening.
|
|
- [Optional] Preferred cloud provider for sync?
|
|
- Best solution: Start with Supabase (Postgres + Auth + Storage) for fastest MVP, then add S3-compatible storage as an optional backend for enterprise/self-hosting.
|
|
- [Core] How long should session memories persist by default?
|
|
- Best solution: 90 days by default with per-session “keep forever” and a global retention slider (7/30/90/365 days).
|
|
- [Core] Should auto-start be opt-in per domain or global?
|
|
- Best solution: Opt-in per domain, with a one-click “trust this site” prompt on first detection.
|
|
- [Optional] What data should be redacted before sync?
|
|
- Best solution: Default to redacting emails, phone numbers, calendar IDs, and detected secrets (API keys/tokens) while letting users add custom redaction rules.
|