Add files via upload
This commit is contained in:
1 parent
d178f6b9ee
commit
2c40eb55ed
11 files changed
+1447
-237
No files matched your search
@@ -1,4 +1,4 @@
|
||||
# ⚙️ MARK XLVIII (48)
|
||||
# ⚙️ MARK XLIX
|
||||
### The Ultimate Cross-Platform Personal AI Assistant — By FatihMakes
|
||||
|
||||
> 📺 **[Watch the full setup video on YouTube](https://www.youtube.com/@FatihMakes)**
|
||||
@@ -9,9 +9,7 @@ A real-time voice AI that can hear, see, understand, and control your computer
|
||||
|
||||
## ✨ Overview
|
||||
|
||||
MARK XLVIII is a direct evolution of Mark XLVII — sharper, faster, and more natural to talk to. This build focused entirely on removing every friction point: the silence while JARVIS "thinks," the awkward double-responses, the interrupts that took four seconds to land, the news results that linked to homepages instead of articles. Everything that made the experience feel like software instead of an assistant has been addressed.
|
||||
|
||||
It's not just an assistant — it's an extension of your digital life.
|
||||
MARK XLIX deepens the personal assistant foundation. Rather than adding more tools, this build focused on making the assistant truly *yours*: it starts with your computer, learns your name, and pays attention to what you're doing. The goal before the plugin era begins is a core that feels alive — not just reactive.
|
||||
|
||||
---
|
||||
|
||||
@@ -27,63 +25,61 @@ It's not just an assistant — it's an extension of your digital life.
|
||||
| 🧠 Persistent Memory | Deeply remembers projects, preferences, and personal context across sessions |
|
||||
| ⌨️ Hybrid Input | Seamlessly switch between keyboard typing and voice commands |
|
||||
| 🌅 Morning Briefing | On first boot: greets you, reads the time, fetches live news headlines, and checks weather |
|
||||
| 🔔 Proactive Check-ins | After 15 minutes of silence JARVIS checks context and offers something genuinely useful — no hardcoded rules, Gemini decides |
|
||||
| 📊 Hardware Monitoring | Continuous CPU, RAM, GPU and temperature telemetry with localized voice alerts when thresholds are breached |
|
||||
| 🔔 Proactive Check-ins | After 15 minutes of silence, checks context and offers something genuinely useful |
|
||||
| 📊 Hardware Monitoring | Continuous CPU, RAM, GPU and temperature telemetry with localized voice alerts |
|
||||
| 🌤️ Weather Report | Live weather data for your city, personalized from memory |
|
||||
| 🗺️ Dynamic Content Panel | Scrollable display layer beneath the HUD that renders web results, news, and search data with timestamps |
|
||||
| 🗺️ Dynamic Content Panel | Scrollable display layer beneath the HUD that renders web results, news, and search data |
|
||||
| 🔍 Multi-Mode Web Search | `news` / `research` / `price` / `compare` / `search` — Gemini Grounded first, DDG fallback |
|
||||
| ⏰ Smart Reminders | OS-native scheduled notifications (Windows Task Scheduler / macOS LaunchAgent / Linux systemd) |
|
||||
| ✈️ Flight Finder | Live flight price and availability lookup |
|
||||
| 🎮 Game Updater | Checks and triggers game updates on demand |
|
||||
| 🎮 Game Updater | Checks and triggers game updates on Steam and Epic Games on demand |
|
||||
| 📂 File Processor | Read, summarize, and answer questions about local files |
|
||||
| 💻 Code Helper | Inline code review, debugging, and generation |
|
||||
| 🌐 Browser Control | Open URLs, navigate tabs, and interact with the browser by voice |
|
||||
| 📨 Send Message | Compose and send messages through integrated messaging apps |
|
||||
| 📨 Send Message | Compose and send messages through WhatsApp, Telegram, and more |
|
||||
| 🎬 YouTube Control | Search, play, and control YouTube playback by voice |
|
||||
| 🖱️ Desktop Control | Taskbar, window management, and desktop-level operations |
|
||||
| 🧑💻 Silent Language Memory | Detects spoken language on first use and saves it silently — all future sessions and briefings adapt automatically |
|
||||
| 🧑💻 Silent Language Memory | Detects spoken language on first use and saves it — all future sessions adapt automatically |
|
||||
| 📱 Remote Dashboard | Control the assistant from your phone via QR code pairing |
|
||||
|
||||
---
|
||||
|
||||
## 🆕 What's New in XLVIII
|
||||
## 🆕 What's New in XLIX
|
||||
|
||||
### ✋ Instant Interrupt — ESC or Button
|
||||
Press **Escape** or click the `INTERRUPT` button to cut JARVIS off mid-sentence and return to listening. Previously, cancelling a response required waiting for the full audio buffer to drain — sometimes four seconds. In Mark XLVIII, audio is split into ~50 ms chunks (2400 bytes each) so the interrupt fires within one slice. The interrupt drains the audio queue, sets a flag, and clears the turn — listening resumes in under 100 ms.
|
||||
### ⚡ Auto-Start on Boot
|
||||
The assistant now registers itself with the operating system's startup system. One click in the UI toggles it on or off. On Windows, it writes to the registry using `pythonw.exe` so no console window ever appears. On macOS it installs a LaunchAgent plist; on Linux a `.desktop` autostart entry. The button reflects the current state every time the app launches.
|
||||
|
||||
### 👁️ Immediate Vision Acknowledgment
|
||||
When you ask JARVIS to look at your screen or camera, it no longer goes silent while processing. The tool now instructs Gemini to immediately say a natural sentence ("Looking at your screen now, sir" / "Ekrana bakıyorum efendim") while the image capture runs. The actual analysis follows as a second response. No more awkward silence.
|
||||
### 🎨 Assistant Customization
|
||||
The assistant is no longer locked to the name "JARVIS". Click `⚙ CUSTOMISE ASSISTANT` in the right panel to change:
|
||||
- **Assistant name** — displayed everywhere in the UI (title bar, header, HUD, log, footer) and injected into the Gemini system prompt so the AI knows its own name
|
||||
- **Your name** — how the assistant addresses you. Leave blank for the default language-aware addressing (`sir` / `efendim`), or set your actual name for a more personal feel
|
||||
|
||||
### 📰 Parallel News Search — First Result Wins
|
||||
News queries now run **Gemini Grounded Search and DuckDuckGo news simultaneously** in two daemon threads. Whichever delivers a valid result first wins; the other is silently discarded. Previously, a Gemini 503 error would stall the search for several seconds before falling back to DDG. Now the fallback happens in parallel — total news fetch time drops to whichever backend is fastest at that moment.
|
||||
Changes take effect immediately without restarting.
|
||||
|
||||
### 🗞️ Real News Articles (Not Homepages)
|
||||
DDG news search was previously using `ddgs.text()`, which returned website homepage URLs for news queries. Mark XLVIII switches to `ddgs.news()`, which returns actual article URLs, titles, snippets, and source names — exactly what you want in a news briefing.
|
||||
### 📋 Clipboard Intelligence
|
||||
Copy any text of 10 or more characters and a floating panel appears at the bottom of the window. Four quick actions — **TRANSLATE**, **SUMMARISE**, **EXPLAIN**, **FIX** — send the copied content directly to the assistant with one click. The panel auto-dismisses after 8 seconds. This turns the clipboard into a silent command channel for anything on your screen.
|
||||
|
||||
### 🌅 Two-Phase Startup Briefing — Runs Concurrently
|
||||
The startup briefing now sends Phase 2 (news fetch) while Phase 1 audio (the greeting) is still playing. Previously, Phase 2 waited for Phase 1 to fully complete. The 1.5-second overlap means the news headline is ready by the time JARVIS finishes saying "Good morning."
|
||||
### ☀ Morning Brief Toggle + Speed Optimization
|
||||
The morning briefing can now be turned on or off with one click from the settings drawer (`⚙` → `☀ MORNING BRIEF: ON/OFF`). Users who don't want a startup briefing can disable it permanently; the setting survives restarts. The briefing itself was also re-engineered: news is now pre-fetched in a background thread the moment the session starts, running in parallel while the greeting plays. By the time the greeting finishes, the results are already ready — no extra Gemini tool-call round-trip needed. Briefing delivery is noticeably faster as a result.
|
||||
|
||||
### 🔁 Smarter Reconnection — Exponential Backoff
|
||||
Network timeouts now use exponential backoff: 3s → 6s → 12s → 60s (capped). Each retry shows a Turkish-language status message in the UI ("Bağlantı kurulamadı — Xs sonra tekrar deneniyor"). Previously, a dropped connection would loop tightly and show no useful information.
|
||||
---
|
||||
|
||||
### 🛡️ Vision Cooldown & Echo Guard
|
||||
Screen capture is guarded against echo loops. If JARVIS speaks about the screen and the microphone picks up its own voice, a secondary `screen_process` call can be triggered. Mark XLVIII blocks duplicate calls with a 4-second cooldown and a `_vision_busy` flag. Both are fully reset when a new session connects — fixing a bug where the cooldown would persist across reconnects and block legitimate requests.
|
||||
## 🗺️ Mark Roadmap
|
||||
|
||||
### 🌐 Language-Aware Address — Never Mixed
|
||||
The system prompt now enforces: **Turkish → efendim**, **English → sir**, never mixed. Mark XLVII's prompt said "Always call sir to user" which caused JARVIS to say "sir" mid-Turkish sentence. Fixed.
|
||||
|
||||
### 🪟 Zero Terminal Windows
|
||||
A subprocess monkey-patch at startup sets `CREATE_NO_WINDOW` on every child process launched by the app. No PowerShell, no CMD, no terminal flash — ever. Applies to all actions including reminders, system commands, and scheduler calls.
|
||||
|
||||
### 🔄 Session State Isolation
|
||||
All transient vision and interrupt flags (`_pending_vision`, `_vision_busy`, `_vision_cam_active`, `_vision_close_pending`, `_interrupted`) are fully reset whenever a new Gemini session connects. Previously, state from a crashed session could carry over and leave JARVIS in a broken state until restart.
|
||||
| Mark | Focus |
|
||||
|---|---|
|
||||
| **XLVIII** | Instant interrupt · parallel news · two-phase briefing · exponential backoff · vision cooldown |
|
||||
| **XLIX** | Auto-start · clipboard intelligence · assistant customization |
|
||||
| **L** | Wake word · proactive system 2.0 · session memory / daily continuity |
|
||||
| **LI+** | Plugin system · email · quiz mode · calorie counter · and more |
|
||||
|
||||
---
|
||||
|
||||
## ⚡ Quick Start
|
||||
|
||||
```bash
|
||||
git clone https://github.com/FatihMakes/Mark-XLVIII.git
|
||||
cd Mark-XLVIII
|
||||
git clone https://github.com/FatihMakes/Mark-XLIX.git
|
||||
cd Mark-XLIX
|
||||
pip install -r requirements.txt
|
||||
python main.py
|
||||
```
|
||||
@@ -106,7 +102,7 @@ python main.py
|
||||
## 🗂️ Project Structure
|
||||
|
||||
```
|
||||
Mark XLVIII/
|
||||
Mark XLIX/
|
||||
├── main.py # Core loop — Gemini Live session, audio I/O, tool dispatch
|
||||
├── ui.py # PyQt6 HUD — waveform, log panel, interrupt button, camera feed
|
||||
├── setup.py # First-run configuration wizard
|
||||
@@ -125,16 +121,16 @@ Mark XLVIII/
|
||||
│ ├── weather_report.py # Live weather data
|
||||
│ ├── flight_finder.py # Flight search
|
||||
│ ├── youtube_video.py # YouTube playback control
|
||||
│ ├── game_updater.py # Game update management
|
||||
│ ├── game_updater.py # Game update management (Steam / Epic)
|
||||
│ ├── code_helper.py # Code review and generation
|
||||
│ ├── dev_agent.py # Developer task agent
|
||||
│ ├── desktop.py # Desktop and taskbar control
|
||||
│ └── proactive.py # Proactive silence-break suggestions
|
||||
│ └── proactive.py # Proactive silence-break suggestions
|
||||
├── memory/ # Persistent key-value memory store
|
||||
├── core/
|
||||
│ └── prompt.txt # JARVIS personality and tool-routing rules
|
||||
│ └── prompt.txt # Assistant personality and tool-routing rules
|
||||
└── config/
|
||||
└── api_keys.json # API key and system configuration
|
||||
└── api_keys.json # API key, OS setting, assistant name, user name
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
Reference in new issue
Block a user