# โš™๏ธ MARK LI (51) ### The Ultimate Cross-Platform Personal AI Assistant โ€” By FatihMakes > ๐Ÿ“บ **[Watch the full setup video on YouTube](https://www.youtube.com/@FatihMakes)** A real-time voice AI that can hear, see, understand, and control your computer โ€” on any OS. Supports Windows, macOS, and Linux. Built on the Gemini Live API for native audio streaming, delivering zero subscriptions and total digital autonomy. --- ## โœจ Overview MARK LI is the final form of the core: an assistant you extend without ever touching its engine. Drop a single plugin file into the `plugins/` folder and JARVIS learns a new skill on the next launch โ€” no code changes, no configuration, no risk. On top of that, the voice itself got smarter: JARVIS now hears the emotion in your voice, knows when you're talking to someone else in the room and stays silent, and can hold one conversation for hours without losing the thread. It's not just an assistant โ€” it's an extension of your digital life. --- ## ๐Ÿš€ Capabilities ### Core Features | Feature | Description | |---|---| | ๐Ÿงฉ Plugin System | Drop a single `.py` file into `plugins/` โ€” JARVIS learns a new skill on next launch | | ๐ŸŽ™๏ธ Real-time Voice | Ultra-low latency conversation in any language via Gemini Live API | | ๐Ÿ’“ Affective Dialog | Hears the emotion in your voice and adapts its tone in response | | ๐Ÿคซ Proactive Audio | Knows when you're not talking to it โ€” background chatter never triggers a reply | | โ™พ๏ธ Unlimited Sessions | Sliding-window context compression โ€” one conversation can last for hours | | ๐Ÿ–ฅ๏ธ System Control | Launch apps, adjust volume/brightness, WiFi, shortcuts, power โ€” all by voice | | ๐Ÿงฉ Autonomous Tasks | High-level planning for complex multi-step goals via agent mode | | ๐Ÿ‘๏ธ Visual Awareness | Real-time screen capture and webcam vision piped into your main Gemini session | | ๐Ÿง  Persistent Memory | Deeply remembers projects, preferences, and personal context across sessions | | โŒจ๏ธ Hybrid Input | Seamlessly switch between keyboard typing and voice commands | | ๐ŸŒ… Morning Briefing | On first boot: greets you, reads the time, recaps yesterday, and fetches live news | | ๐Ÿ”” Proactive 2.0 | Time-aware, context-aware check-ins โ€” knows the time of day, your projects, and what you've been discussing | | ๐Ÿ—“๏ธ Session Memory | Summarises each conversation and mentions it naturally next morning โ€” consumed after use, never repeats | | ๐Ÿ‘๏ธโ€๐Ÿ—จ๏ธ Background Monitoring | User-configured topic watching โ€” checks for new headlines once a day and alerts naturally | | ๐Ÿ“Š Hardware Monitoring | Continuous CPU, RAM, GPU and temperature telemetry with localized voice alerts | | ๐ŸŒค๏ธ Weather Report | Live weather data for your city, personalized from memory | | ๐Ÿ—บ๏ธ Dynamic Content Panel | Scrollable display layer beneath the HUD that renders web results, news, and search data | | ๐Ÿ” Multi-Mode Web Search | `news` / `research` / `price` / `compare` / `search` โ€” Gemini Grounded first, DDG fallback | | โฐ Smart Reminders | OS-native scheduled notifications (Windows Task Scheduler / macOS LaunchAgent / Linux systemd) | | โœˆ๏ธ Flight Finder | Live flight price and availability lookup | | ๐ŸŽฎ Game Updater | Checks and triggers game updates on Steam and Epic Games on demand | | ๐Ÿ“‚ File Processor | Read, summarize, and answer questions about local files | | ๐Ÿ’ป Code Helper | Inline code review, debugging, and generation | | ๐ŸŒ Browser Control | Open URLs, navigate tabs, and interact with the browser by voice | | ๐Ÿ“จ Send Message | Compose and send messages through WhatsApp, Telegram, and more | | ๐ŸŽฌ YouTube Control | Search, play, and control YouTube playback by voice | | ๐Ÿ–ฑ๏ธ Desktop Control | Taskbar, window management, and desktop-level operations | | ๐Ÿง‘โ€๐Ÿ’ป Silent Language Memory | Detects spoken language on first use โ€” all future sessions adapt automatically | | ๐Ÿ“ฑ Remote Dashboard | Control the assistant from your phone via QR code pairing | | โšก Auto-Start on Boot | Registers with the OS startup system (registry / LaunchAgent / .desktop) | | ๐Ÿ“‹ Clipboard Intelligence | Copy any text โ†’ floating panel with Translate / Summarise / Explain / Fix | | ๐ŸŽจ Assistant Customization | Change the assistant name and your name from the UI โ€” takes effect immediately | --- ## ๐Ÿ†• What's New in Mark LI ### ๐Ÿงฉ Plugin System โ€” Extend JARVIS Without Touching a Single Core File The headline feature of Mark LI, and the reason it's the final architecture version. Every new capability from now on ships as a single `.py` file: 1. Download a plugin file (e.g. `calorie_counter.py`) 2. Drop it into the `plugins/` folder 3. Restart JARVIS โ€” done. The skill is live, by voice, in any language. Each plugin declares its own Gemini tool schema and logic in one file. The engine auto-discovers it at startup, registers it with the Live session, and lists it in the new **๐Ÿงฉ Plugin Manager** panel where every plugin gets its own persistent ON/OFF toggle. Safety is built in at three layers: a broken or badly written plugin can **never** crash JARVIS โ€” it simply shows up as "BROKEN" in the manager with the error explained, while every other tool and plugin keeps working. Name collisions with core tools are detected and rejected automatically. Want to write your own? Copy `plugins/_template.py` and fill in two things: the `PLUGIN` dict and the `run()` function. ### ๐Ÿ’“ Affective Dialog โ€” JARVIS Hears How You Feel Powered by Gemini Live's native audio understanding, JARVIS now picks up the emotion in your voice โ€” excitement, frustration, fatigue โ€” and adapts its own tone in response. Late-night tired questions get calm answers; excited announcements get energy back. ### ๐Ÿคซ Proactive Audio โ€” Knows When You're Not Talking to It The biggest quality-of-life upgrade for an always-listening assistant: JARVIS can now tell when speech isn't addressed to it. Talking to someone in the room, taking a phone call, TV in the background โ€” it stays silent instead of interjecting. No wake word needed, no accidental replies. ### โ™พ๏ธ Unlimited Session Length โ€” The Conversation Never Dies Sliding-window context compression means the Live session no longer terminates when the context window fills up. Combined with session resumption, JARVIS holds one continuous conversation for hours without losing the thread. All three Live API upgrades degrade gracefully: if the preview API ever rejects them, JARVIS automatically reconnects with the standard configuration โ€” users never see a crash. --- ## ๐Ÿ—บ๏ธ Mark Roadmap | Mark | Focus | |---|---| | **XLVIII** | Instant interrupt ยท parallel news ยท two-phase briefing ยท exponential backoff ยท vision cooldown | | **XLIX** | Auto-start ยท clipboard intelligence ยท assistant customization | | **L** | Session memory ยท background monitoring ยท proactive 2.0 ยท instant vision ยท parallel news search | | **LI** | Plugin system ยท affective dialog ยท proactive audio ยท unlimited sessions | | **LII+** | Plugin files: email ยท quiz mode ยท calorie counter ยท calendar ยท and more | --- ## โšก Quick Start ```bash git clone https://github.com/FatihMakes/Mark-LI.git cd Mark-LI pip install -r requirements.txt python main.py ``` > โš ๏ธ **Installation Note:** Some OS-specific dependencies are not bundled in `requirements.txt` to keep the repo lightweight. If you hit a `ModuleNotFoundError`, install the missing package with `pip install `. --- ## ๐Ÿ“‹ Requirements | Requirement | Details | | --- | --- | | **OS** | Windows 10/11, macOS, or Linux | | **Python** | 3.11 or 3.12 | | **Microphone** | Required for voice interaction | | **API Key** | Free Gemini API key (`config/api_keys.json`) | --- ## ๐Ÿ—‚๏ธ Project Structure ``` Mark LI/ โ”œโ”€โ”€ main.py # Core loop โ€” Gemini Live session, audio I/O, tool dispatch โ”œโ”€โ”€ ui.py # PyQt6 HUD โ€” waveform, log panel, plugin manager, camera feed โ”œโ”€โ”€ setup.py # First-run configuration wizard โ”œโ”€โ”€ plugins/ โ”‚ โ””โ”€โ”€ _template.py # Copy this to write a new plugin โ€” one file, drop in, done โ”œโ”€โ”€ actions/ โ”‚ โ”œโ”€โ”€ web_search.py # Gemini + DDG parallel search (news, research, price, compare) โ”‚ โ”œโ”€โ”€ screen_processor.py # Screen capture & webcam vision via Gemini Live โ”‚ โ”œโ”€โ”€ background_monitor.py # User-configured topic watching โ€” daily DDG check, no crypto โ”‚ โ”œโ”€โ”€ proactive.py # Proactive 2.0 โ€” time/context/rotation-aware check-ins โ”‚ โ”œโ”€โ”€ reminder.py # OS-native scheduled notifications โ”‚ โ”œโ”€โ”€ system_monitor.py # CPU / RAM / GPU / temperature telemetry โ”‚ โ”œโ”€โ”€ computer_settings.py # Volume, brightness, WiFi, power โ”‚ โ”œโ”€โ”€ computer_control.py # Keyboard shortcuts, mouse, window management โ”‚ โ”œโ”€โ”€ open_app.py # Application launcher โ”‚ โ”œโ”€โ”€ browser_control.py # Web browser control โ”‚ โ”œโ”€โ”€ file_controller.py # File system operations โ”‚ โ”œโ”€โ”€ file_processor.py # Document reading and summarization โ”‚ โ”œโ”€โ”€ send_message.py # Messaging integration โ”‚ โ”œโ”€โ”€ weather_report.py # Live weather data โ”‚ โ”œโ”€โ”€ flight_finder.py # Flight search โ”‚ โ”œโ”€โ”€ youtube_video.py # YouTube playback control โ”‚ โ”œโ”€โ”€ game_updater.py # Game update management (Steam / Epic) โ”‚ โ”œโ”€โ”€ code_helper.py # Code review and generation โ”‚ โ”œโ”€โ”€ dev_agent.py # Developer task agent โ”‚ โ””โ”€โ”€ desktop.py # Desktop and taskbar control โ”œโ”€โ”€ memory/ โ”‚ โ”œโ”€โ”€ memory_manager.py # Load/save long_term.json โ€” sessions, monitors, identity โ”‚ โ””โ”€โ”€ long_term.json # Persistent store: identity, preferences, projects, sessions, monitors โ”œโ”€โ”€ core/ โ”‚ โ”œโ”€โ”€ prompt.txt # Assistant personality and tool-routing rules โ”‚ โ””โ”€โ”€ plugin_loader.py # Plugin engine โ€” discovery, validation, crash isolation โ””โ”€โ”€ config/ โ””โ”€โ”€ api_keys.json # API key, OS setting, assistant name, user name ``` --- ## โš ๏ธ License Personal and non-commercial use only. Licensed under **[Creative Commons BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/)**. --- ## ๐Ÿ‘ค Connect with the Creator Engineered by a developer building a real-world JARVIS-style assistant. โญ **Star the repository to support the journey to Mark 100.** | Platform | Link | | --- | --- | | YouTube | [@FatihMakes](https://www.youtube.com/@FatihMakes) | | Instagram | [@fatihmakes](https://www.instagram.com/fatihmakes) |