Skip to content

About

No description, website, or topics provided.

Resources

Contributing

Security policy

Stars

224 stars

Watchers

16 watching

Forks

Latest commit

 

History

1,125 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Wingman AI Core

Your AI crew for Star Citizen and every other game. Talk to your Wingmen in plain words. They press the keys, look things up for you and react on their own to what happens in the game.

Website and download: wingman-ai.com · Community: Discord

Never Fly Solo Again: Wingman AI 3 for Star Citizen

Wingman AI runs next to your game. You talk, an AI model of your choice understands what you mean, and your Wingman answers in its own voice and presses the keys you would press yourself. Each Wingman is a character with a backstory, a voice, commands and skills. You can set all of it up in the client without being a developer, and developers can extend everything in Python.

This repository is Wingman AI Core, the open source backend. It records your voice, talks to the AI providers, runs commands, skills and MCP servers, and exposes all of it through a REST and WebSocket API. Our client for Windows, macOS and Linux is the app on top of it.

Things you can say:

  • "Power up my ship!" and your Wingman runs the macro that does it.
  • "It's raining, I can't see anything." and it calls the command you named TurnFlashlightOn. No exact phrases to learn.
  • "Where do I get the best price for Laranite?" and it looks it up live.
  • "What did we do in our last session?" and it remembers.
  • "Talk like a grumpy pirate from now on."
  • "What do you see on my screen right now?"
  • "Set a timer for ten minutes."

Chat with a Wingman Commands Settings

Contents

Features

Talk to it

  • Push-to-talk or voice activation. Voice activation uses Silero VAD to tell speech from noise.
  • Speaks your language: English, German, French, Spanish, Italian, Portuguese or Dutch, picked once for speech recognition, answers and voice. For any other language, choose "Other" and name it.
  • Says it right: abbreviations, units and numbers are spoken as words in your language, plus your own pronunciation rules and per-game lists (e.g. Star Citizen's aUEC or QT).
  • Voice effects like radio, AI or the sound of a small or large room, and audio markups ("emotions") such as a laugh or a sigh with voices that support them. Both work with every TTS provider.

Wingmen with a character

  • Backstory, voice and AI model per Wingman. Describe its role in plain words, or tweak every part of the prompt.
  • Persistent memory: Wingmen remember facts about you across sessions, and keep track of missions and plans until they are done. You can see and change what they remember, and ask them to forget.
  • Avatar studio: the client paints an avatar from the backstory in a style of your choice.
  • Filler lines: while a tool runs, your Wingman says a short line in character instead of going silent.

Commands

  • Smart commands: the AI picks the right command from what you said. Users don't need to say exact phrases.
  • Instant activation: exact phrases fire a command right away, without an AI round-trip. The Quick Commands skill learns these phrases for the commands you use often.
  • Keyboard, mouse and joystick: record single keys, macros and mouse movement by pressing the buttons.
  • A command can also play a sound file from the Sound Library or answer with a line you wrote.

HUD overlay

A transparent overlay on top of your game, on Windows, macOS and Linux. It is part of Core, no skill or MCP server needed. Your Wingmen show their replies there, pin info panels like your current trade run, and add progress bars and timers. HUD profiles lay it out per game, and you place them by dragging them on screen. Details in docs/hud.md. Community member JayMatthew created it.

Star Citizen log reader

Star Citizen writes a log file while you play. The log reader is part of Core: it follows that file and passes on missions, armistice zones, CrimeStat, injuries, your hangar and trades, so your Wingman knows what happens without you saying a word. It only reads the log file: no mods, nothing in the game's memory. Community member Mallachi created it, together with the Star Citizen Events and Accountant skills.

HUD overlay in Star Citizen Log reader in Star Citizen

Skills

Skills are Python modules that give a Wingman new abilities. Grant one to a Wingman and it decides when to use it. Built-in skills:

  • General
    • API Request: sends HTTP requests to any API
    • Audio Device Changer: routes a Wingman's voice to its own audio device
    • Auto Screenshot: takes screenshots on command or when you get excited
    • Control Windows: launches, switches, arranges and closes apps, and uses the clipboard
    • File Manager: reads, writes and searches files, handles ZIP files and reads text aloud
    • Image Generation: creates images in 13 styles, in square, portrait or landscape, and changes an earlier image or one you attach ("the same character, but ...")
    • Microphone Status: shows the live mic state on the HUD
    • Quick Commands: learns instant activation phrases for the commands you use often
    • Radio Chatter: plays random radio chatter with voices you pick
    • Spotify: plays songs, artists and playlists and controls playback
    • Thinking Sound: plays a sound while the AI thinks
    • Timer: delays reminders, commands and other actions
    • Typing Assistant: types text into any app
    • Vision AI: looks at your screen, reads text and describes what it sees
    • Voice Changer: changes the voice of your Wingman automatically
  • Star Citizen
    • Star Citizen Events: your Wingman reacts in character to what the log reader sees, and knows where you are and which missions you have
    • Star Citizen Accountant: books trades and rewards from the game log and tracks money, cargo, refining, fleet and plans, with a dashboard for your computer or phone
    • UEX Corp: trade routes, commodity prices, locations, ships and items from UEX community data
  • Other games (Windows only)
    • MSFS2020 Control: controls Microsoft Flight Simulator 2020/2024 and reads its data via SimConnect
    • ATS Telemetry: live data from American Truck Simulator and Euro Truck Simulator 2, with a dispatch mode that comments on what happens

You can write your own skill, or let an AI coding agent like Claude Code, Codex or Cursor write it for you: see Creating Custom Skills.

MCP servers

Wingman AI is a full MCP (Model Context Protocol) client for local and remote servers, over HTTP, STDIO and SSE. Connect a Wingman to any MCP server by its URL, or build your own in any programming language. Skills are Python modules inside Core, MCP servers are separate programs.

Every subscription plan, Free included, gets our hosted servers:

  • Web Search: Brave and Tavily search and page extraction, no API key needed
  • Date & Time: time zones and date calculations
  • StarHead: Star Citizen ships, components, locations and trade routes
  • VerseTime: local time, sunrise and sunset at Star Citizen locations
  • Galactapedia: Star Citizen lore from RSI's Galactapedia
  • Elite Dangerous: systems, stations, services, commodity prices and engineering
  • No Man's Sky: items, refiner and cooking recipes, expeditions
  • Perplexity: AI search with citations, with your own API key
  • ElevenLabs: images, video, speech, transcription and voices, signed in with your own ElevenLabs account (docs/mcp-oauth.md)

AI providers

Speech-to-text, the AI model and text-to-speech are three separate steps. Take the Wingman Subscription for all of them, pick a provider for each, or run all three on your own computer.

  • AI models: OpenAI, Google (Gemini), Mistral, Groq, OpenRouter, Cerebras, Perplexity, xAI (Grok), any OpenAI-compatible local LLM (llama.cpp, Ollama, LM Studio), and the Wingman Subscription
  • Speech-to-text: Parakeet (local, the default, on the GPU with NVIDIA CUDA), Parakeet on your own server, and the Wingman Subscription
  • Text-to-speech:
    • Pocket TTS (local and free, runs on the CPU, clones a voice from a few seconds of audio, with ready-made voices from native speakers: 49 German, 21 English, 20 French, 15 Spanish, 20 Italian, 20 Portuguese and 20 Dutch)
    • Edge TTS (free), OpenAI, ElevenLabs, Hume, Inworld
    • Azure and Inworld through the Wingman Subscription
    • any OpenAI-compatible server, e.g. a remote Pocket TTS or XVASynth
  • Image generation: OpenAI and the Wingman Subscription
  • Support model for background work like memory, summaries and shortening long tool responses. It runs in our cloud by default, locally with llama.cpp (Qwen 3.5 4B or Gemma 4 E2B), or on your own llama.cpp server.

For developers

  • REST API with an OpenAPI spec at http://127.0.0.1:49111/docs, and WebSockets to send and receive messages from clients, games or devices
  • File-based configuration: one folder per game or use case, one YAML file per Wingman. No database needed.
  • Custom Wingmen and skills in Python, with a guide written for AI coding agents

Star Citizen

Wingman AI is deeply rooted in the Star Citizen community, and most of our members play it. On the first start the client asks whether you play it. Yes switches on everything below for the Computer and ATC Wingmen:

  • the HUD overlay and the log reader
  • the Star Citizen Events and Star Citizen Accountant skills
  • live data from UEX Corp and StarHead, local times from VerseTime and lore from Galactapedia
  • Star Citizen pronunciation and transcription lists, so terms like aUEC or QT come out right

You can change the answer, the game folder and every single feature later in Settings > Star Citizen.

Is it just for Star Citizen?

No. Wingman AI is a universal tool that runs alongside any game or program, as your copilot or as a character to roleplay with. Have air traffic control in Microsoft Flight Simulator, or talk to Shadowheart in Baldur's Gate 3 in her own cloned voice.

It does not hook into games. It presses system-wide keys, which of course reach the game, and in Star Citizen it reads the log file the game writes anyway. It never reads or changes a game's memory. We're not affiliated with CIG or Star Citizen in any way.

The video that started it all, our first prototype from 2023:

Early prototype on Wingman in Star Citizen

Providers & cost

Wingman AI Core is open source and free, and so is the client. The AI providers you use might not be. That's why we offer the Wingman Subscription: one account for chat models, speech recognition, voices and image generation, with a monthly usage allowance instead of pay-per-use bills and no API keys.

  • Free has almost every feature: as many Wingmen as you want, commands, skills, MCP servers, the HUD, the avatar studio and voices in every language, with a budget that refills every month. Only Voice Activation needs Pro or Ultra.
  • Pro gives you 10x the usage of Free, Ultra 30x. Both add Voice Activation and more voices, and Ultra includes every voice.
  • Tell a friend: share your invite link, and when a friend subscribes, you both get a Usage Reset.

Compare the plans on wingman-ai.com.

You don't need the subscription to use Wingman AI: Parakeet, a local LLM and Pocket TTS run all three steps on your own computer, and you can bring your own API keys for the providers below.

OpenAI

OpenAI charges by usage: you pay for each call to their APIs instead of a flat fee. See their pricing. To create an API key:

  • Sign up or log in at platform.openai.com. You don't need ChatGPT Plus to use Wingman AI.
  • Go to "Billing" and add a payment method. We recommend setting a low usage limit to avoid unexpected costs.
  • Go to API keys and create one. Copy it! If you lose it, you can always create a new one.

ElevenLabs

ElevenLabs has great voices, and Wingman AI can generate instant sound effects with their API. You can clone any voice with 3 minutes of clean audio, e.g. your friend, an actor or a recording of an NPC in your game.

If you're interested in the service, please consider using our referral link. It costs you nothing extra and supports Wingman AI. We get 22% of all payments in your first year. Thank you!

Create your account, set up your payment method and create an API key. Enter that API key in Wingman AI when asked.

Inworld

Inworld has high-quality voices with audio markups for emotions (anger, joy, sadness and more). It is included in the Wingman Subscription: Pro gets two Inworld voices per language, Ultra gets all of them. With your own API key from Inworld AI you can also clone voices, at a much lower price than ElevenLabs.

Hume

Similar to ElevenLabs and Inworld, a bit less polished and quite affordable.

Edge TTS

Microsoft Edge TTS is free and needs no API key, but its voices sound less natural than the others. Each voice speaks only one language. When you switch the language, Wingman picks a matching voice for you.

Local LLMs

Use the "Local LLM" provider and connect any LLM with an OpenAI-compatible API, for example a llama.cpp server, Ollama or LM Studio.

Local text-to-speech

Pocket TTS is bundled with Wingman AI and works out of the box. It runs on the CPU and clones a voice from a few seconds of audio. To run it on another machine, e.g. a stronger GPU server, use the Pocket TTS OpenAI Streaming Server and connect to it with the "OpenAI-compatible" TTS provider.

XVASynth (Windows only) is also supported and installable via Steam.

Installing Wingman AI

You install Wingman AI once. The client updates itself and Core when a new version comes out.

Windows

  • Download the installer of the latest version from wingman-ai.com.
  • CUDA is bundled. With an NVIDIA GPU from the GTX 16xx / RTX 20xx series or newer and driver 580 or newer, local speech recognition (Parakeet) runs on the GPU without extra installation steps. Older GPUs and drivers use the CPU.
  • Install it to a directory of your choice and start the client Wingman AI.exe.
    • The client will auto-start Wingman AI Core.exe in the background

If that doesn't work for some reason, try starting Wingman AI Core.exe manually and check the terminal or your logs directory for errors.

If you're a developer, you can also run from source. This way you can preview our latest changes on the develop branch and debug the code.

macOS

  • Download the .dmg from wingman-ai.com and move Wingman AI to your Applications folder.
  • It needs a Mac with Apple Silicon (M1 or newer) and macOS 14 (Sonoma) or newer.
  • On the first start, allow access to the microphone. For hotkeys and push-to-talk, also allow Accessibility in System Settings > Privacy & Security.
  • Not available on macOS: XVASynth and the two Windows-only skills (ATS Telemetry, MSFS2020 Control).

You can also run it from source.

Linux

Linux support is still experimental. We need people who use it to help us make it better, so please report what doesn't work on Discord or in an issue.

  • Download the AppImage (x86_64) from wingman-ai.com, make it executable (chmod +x WingmanAI_*.AppImage) and start it.
  • It needs a distribution from 2024 or newer (glibc 2.39): Ubuntu 24.04, Fedora 40, Debian 13, current Arch, CachyOS, Nobara or newer.
  • Install PortAudio and the PipeWire ALSA plugin. Wingman uses the system's audio libraries, so your headset and the PipeWire/Pulse devices show up:
    • Fedora, Nobara: sudo dnf install portaudio pipewire-alsa
    • Ubuntu, Debian: sudo apt install libportaudio2 pipewire-alsa libfuse2t64
    • Arch, CachyOS: sudo pacman -S portaudio pipewire-alsa
  • For hotkeys and push-to-talk, add yourself to the input group, then log out and back in: sudo usermod -aG input $USER. This works on X11 and Wayland.
  • The HUD overlay needs an X display (X11, or XWayland in a Wayland session) and a compositor, which every desktop has.
  • Not available on Linux: XVASynth and the two Windows-only skills (ATS Telemetry, MSFS2020 Control).

If Wingman does not start, run the AppImage from a terminal. Core's messages, such as a missing PortAudio, are printed there.

Who are these Wingmen?

Our default Wingmen are examples and starting points for your own. You can change them in the client or add as many Wingmen of your own as you like.

  • Computer is your ship's board computer in Star Citizen. It comes with keybindings and commands for all ship systems.
  • ATC is air traffic control. It handles your calls to stations, like landing requests, departure clearances and emergencies, and stays in character.
  • Clippy lives in the "General" configuration, for everything outside of games.

By default, they use the Wingman Subscription.

How a turn works

  1. Your speech is transcribed by the configured speech-to-text provider.
  2. The transcript goes to the AI model, together with the Wingman's backstory, its memory and what is happening in the game. The model answers with text and maybe tool calls.
  3. Wingman AI Core runs the tool calls: commands, skills or MCP servers.
  4. The answer is read out in the Wingman's voice.
  5. Clients connected to Core see every step live.

The AI picks a command by its name, so give each command a good one (e.g. Request landing permission).

StarHead

StarHead

StarHead is a community project with a huge database of Star Citizen trade items, shop inventories and prices, kept up to date by its players. Computer and ATC use it as an MCP server. Ask for the best trade route, and your Wingman asks for your ship, location and budget, calls the StarHead API and reads the result back to you. You can ask follow-up questions, like the next stop or a route from a different location.

For updates, visit the StarHead website or follow @KNEBEL on Twitch and YouTube.

Noteworthy community projects

Can I configure Wingman AI Core without using your client?

Yes, you can! You can edit all the configs in the configs folder of your version directory:

  • Windows: %APPDATA%\ShipBit\WingmanAI\[version]\configs
  • macOS: ~/Library/Application Support/WingmanAI/[version]/configs
  • Linux: ~/.local/share/WingmanAI/[version]/configs

[version] uses underscores, e.g. 3_2_9.

The YAML configs are very indentation-sensitive, so please be careful.

There is no hot reloading, so you have to restart Wingman AI Core after you made manual changes to the configs.

Directory/file-based configuration

Use these naming conventions to create different configurations for different games or scenarios:

  • any subdirectory in your config dir is a "configuration" or use case. Do not use special characters.
    • The configuration that is launched on start is stored in context.yaml as default_config.
  • Inside of a configuration directory, you can create different wingmen by adding [name].yaml files. Do not use special characters.
    • .[name].yaml (dot): marks the Wingman as "hidden" and skips it in the UI and on start, e.g. .Computer.yaml.
    • [name].png (image): Sets an avatar for the Wingman in the client, e.g. StarHead.png. The client's avatar studio paints one from the backstory in a style of your choice and keeps every variant in generated_files/AvatarStudio/[wingman name]/.

There are a couple of other files and directories in the config directory that you can use to configure Wingman AI.

  • defaults.yaml - contains the default settings for all Wingmen. This is merged with the settings of the individual Wingmen at runtime. Specific wingman settings always override the defaults. Once a wingman is saved using the client, it contains all the settings it needs to run and will no longer fallback to the defaults.
  • settings.yaml - contains user settings like the selected audio input and output devices
  • secrets.yaml - contains the API keys for different providers.
  • mcp.yaml - the MCP server connections
  • context.yaml - the default configuration and the template configurations and Wingmen you deleted, so they are not created again

Access secrets in code by using secret_keeper.py. You can access everything else with config_manager.py.

Does it support my language?

Wingman has one language setting for speech recognition, answers and voice. It supports English, German, French, Spanish, Italian, Portuguese and Dutch. Pick it once in the settings, and Wingman sets up Parakeet, the answer language and the Pocket TTS voices to match.

For any other language, choose "Other" and name your language. Wingman then lets speech recognition detect the language, tells the AI to answer in it and uses a voice provider that has voices for it.

You can trigger commands in your language without translating their name. The AI understands what you mean.

Develop with Wingman AI

Are you ready to build your own Wingman or implement new features to the framework?

Please follow our guides to setup your dev environment:

Run the tests with pip install -r requirements-dev.txt and python -m pytest.

Creating Custom Skills

Skills are small Python modules. Put your own into the custom_skills folder next to your configs folder, and Wingman AI loads it on the next start.

  • Guide for AI coding agents: let Claude Code, Codex or Cursor write the skill for you. Describe what it should do, and the agent also tells you when an MCP server is the better choice.
  • Migrating to v3: port a skill written for an older Skill API.
  • Skills Developer Guide: the complete guide, including:
    • How skills work and are discovered
    • Tool-based vs hook-based skills
    • Skill vs MCP decision guide
    • Available custom property types
    • Bundling dependencies
    • Best practices and examples

If you want to read some code first and understand how it all works, we recommend you start here (in this order):

  • http://127.0.0.1:49111/docs - The OpenAPI (ex: Swagger) spec
  • wingman_core.py - most of the public API endpoints that Wingman AI exposes
  • The config files in your config directory to get an idea of what's configurable.
  • wingmen/wingman.py - the base class for all Wingmen
  • wingmen/open_ai_wingman.py - derived from Wingman, using all the providers
  • services/tower.py - the factory that creates Wingmen

If you're planning to develop a major feature or new integration, please contact us on Discord first and let us know what you're up to. We'll be happy to help you get started and make sure your work isn't wasted because we're already working on something similar.

Contributing

We welcome contributions! Please read our Contributing Guide before submitting pull requests. Key points:

  • PRs target the develop branch. Link an issue if there is one
  • PRs are squash-merged with linear history (rebase onto develop, no merge commits)
  • At least one approving review is required, and the tests have to pass
  • Use the Bug Report or Feature Request templates when creating issues

Acknowledgements

Thank you so much for your support. We really appreciate it!

Open Source community

Wingman makes use of other Open Source projects internally. We would like to thank their creators for their great work and contributions to the Open Source community. We use them without changes, except for keyboard and mouse, which ship as copies in this repository with our own fixes for macOS and Linux.

  • edge-tts - LGPL-3.0
  • elevenlabslib - MIT, © 2023 lugia19
  • FastAPI - MIT, © 2018 Sebastián Ramírez
  • google-genai - Apache-2.0
  • httpx - BSD 3, © 2019 Encode OSS Ltd.
  • huggingface_hub - Apache-2.0
  • hume - MIT, © Hume
  • keyboard - MIT, © 2016 BoppreH
  • llama.cpp - MIT, © 2023-2026 The ggml authors. Wingman downloads the official llama-server binaries to run the support model locally.
  • mcp - MIT, © 2024 Anthropic, PBC
  • mouse - MIT, © 2016 BoppreH
  • num2words - LGPL-2.1
  • numpy - BSD 3, © 2005-2025 NumPy Developers
  • onnx-asr - MIT, © 2025 Ilya Stupakov
  • ONNX Runtime - MIT, © Microsoft Corporation
  • openai - Apache-2.0
  • packaging - Apache/BSD, © Donald Stufft and individual contributors
  • pedalboard - GPL-3.0, © 2021-2023 Spotify AB
  • Pillow - MIT-CMU, © 1997-2011 Secret Labs AB, © 1995-2011 Fredrik Lundh and contributors
  • platformdirs - MIT, © 2010-202x plaformdirs developers
  • pocket-tts - MIT
  • pydantic - MIT, © 2017 to present Pydantic Services Inc. and individual contributors
  • pygame - LGPL-2.1
  • pydirectinput-rgx - MIT, © 2022 dev@reggx.eu, 2020 Ben Johnson
  • pyinstaller - extended GPL 2.0, © 2010-2023 PyInstaller Development Team
  • PyTorch and torchaudio - BSD 3, © 2016- Facebook, Inc and contributors
  • PyYAML - MIT, © 2017-2021 Ingy döt Net, 2006-2016 Kirill Simonov
  • RapidFuzz - MIT, © 2020-present Max Bachmann, © 2011 Adam Cohen
  • regex - Apache-2.0
  • requests - Apache-2.0
  • scipy - BSD 3, © 2001-2002 Enthought, Inc. 2003-2023, SciPy Developers
  • sentry-python - MIT, © 2018 Functional Software, Inc. dba Sentry
  • sounddevice - MIT, © 2015-2023 Matthias Geier
  • soundfile - BSD 3, © 2013 Bastian Bechtold
  • tiktoken - MIT, © 2022 OpenAI, Shantanu Jain
  • uvicorn - BSD 3, © 2017-present, Encode OSS Ltd. All rights reserved.

AI models

Wingman ships or downloads these models. We use them without changes.

Voice recordings

Pocket TTS clones voices from recordings of native speakers. Wingman ships short excerpts (about 10 seconds each) in templates/pocket_tts/voices/. We thank the readers of LibriVox, who dedicate their recordings to the public domain, and Thorsten Müller for Thorsten-Voice (CC0 1.0). Where a reader goes by a user name, the voice has a first name of its own. The reader is named next to it.

Individual persons

This list will inevitably remain incomplete. If you miss your name here, please let us know in Discord or via Patreon.

Special thanks

  • JayMatthew aka SawPsyder, @teddybear082, @Thaendril and @Xul for outstanding moderation in Discord, constant feedback and valuable Core & Skill contributions
  • @lugia19 for developing and improving the amazing elevenlabslib.
  • Knebel who helped us kickstart Wingman AI by showing it on stream and grants us access to the StarHead API for Star Citizen.
  • @Zatecc from UEX Corp who supports our community developers and Wingmen with live trading data for Star Citizen using the UEX Corp API.

About

No description, website, or topics provided.

Resources

Contributing

Security policy

Stars

224 stars

Watchers

16 watching

Forks

Releases

Sponsor this project

Packages

Used by

Contributors

Languages