feat: Enhance audio capture and monitoring features
- Added "audioCapture" permission to manifest for microphone access. - Introduced DeepSeek as a new AI provider option in the side panel. - Implemented a capture mode selection (tab-only, mic-only, mixed) in the side panel. - Added options to enable/disable the extension and auto-open the assistant window. - Integrated a mic monitor feature with live input level visualization. - Included buttons for requesting microphone permission and granting tab access. - Updated styles for new sections and mic level visualization. - Enhanced model fetching logic to support DeepSeek and improved error handling.
This commit is contained in:
78
Plans_and_Todo.md
Normal file
78
Plans_and_Todo.md
Normal file
@@ -0,0 +1,78 @@
|
||||
# Personal Browser Companion - Plans & To-Do
|
||||
|
||||
## Goals
|
||||
- Start local-first with an option to sync to cloud.
|
||||
- Online-only operation (LLM required for decisions).
|
||||
- Auto-start mode during meetings.
|
||||
- Integrations: calendar, email, Discord, Nextcloud.
|
||||
|
||||
## Phase Plan
|
||||
|
||||
### Phase 1: Local MVP (Foundation)
|
||||
- Local storage for sessions, summaries, and user profile.
|
||||
- Meeting/interview modes with manual start and overlay UI.
|
||||
- Basic memory retrieval: recent session summaries + user profile.
|
||||
- Audio capture + STT pipeline (mic + tab) and transcript display.
|
||||
- Privacy controls: store/forget, per-session toggle.
|
||||
|
||||
### Phase 2: Smart Auto-Start
|
||||
- Detect meeting tabs (Google Meet, Zoom, Teams) and prompt to start.
|
||||
- Auto-start rules (domain allowlist, time-based, calendar hints).
|
||||
- Lightweight on-device heuristics for meeting detection.
|
||||
|
||||
### Phase 3: Cloud Sync (Optional)
|
||||
- Opt-in cloud sync for memory + settings.
|
||||
- Conflict resolution strategy (last-write wins + merge for summaries).
|
||||
- Encryption at rest, user-controlled delete/export.
|
||||
|
||||
### Phase 4: Integrations (MCP)
|
||||
- Calendar: read upcoming meetings, attach context.
|
||||
- Email: draft follow-ups, summaries.
|
||||
- Discord: post meeting summary or action items to a channel.
|
||||
- Nextcloud: store meeting notes, transcripts, and attachments.
|
||||
|
||||
## MVP To-Do (Local)
|
||||
|
||||
### Core
|
||||
- Define memory schema (profile, session, summary, action items).
|
||||
- Implement local RAG: index summaries + profile into embeddings.
|
||||
- Add session lifecycle: start, pause, end, summarize.
|
||||
|
||||
### Audio + STT
|
||||
- Implement reliable STT for tab audio (server-side if needed).
|
||||
- Keep mic-only STT as fallback.
|
||||
- Add device selection + live mic monitor.
|
||||
|
||||
### UI/UX
|
||||
- Overlay controls: resize, hide/show, minimize.
|
||||
- Auto-start toggle in side panel.
|
||||
- Session summary view with “save to memory” toggle.
|
||||
|
||||
### Privacy
|
||||
- Per-session storage consent prompt.
|
||||
- “Forget session” button.
|
||||
|
||||
## Integration To-Do (MCP)
|
||||
|
||||
### MCP Server Options
|
||||
- Build a local MCP server as a bridge for integrations.
|
||||
- Use MCP tool registry for calendar/email/Discord/Nextcloud.
|
||||
|
||||
### Calendar
|
||||
- Read upcoming meetings and titles.
|
||||
- Auto-attach relevant context packs.
|
||||
|
||||
### Email
|
||||
- Generate follow-up drafts from summary + action items.
|
||||
|
||||
### Discord
|
||||
- Post meeting summary/action items to a selected channel.
|
||||
|
||||
### Nextcloud
|
||||
- Upload meeting notes and transcripts.
|
||||
|
||||
## Open Questions
|
||||
- Preferred cloud provider for sync?
|
||||
- How long should session memories persist by default?
|
||||
- Should auto-start be opt-in per domain or global?
|
||||
- What data should be redacted before sync?
|
||||
Reference in New Issue
Block a user