Ailly

Universal On-Device Vision & Accessibility Layer

A browser-native, privacy-first platform that reimagines accessibility by combining real-time local vision AI with web-wide accessibility translation. Powered by WebGPU for on-device ML inference at interactive latencies.

🚀 Features

Vision AI Workstation

• Real-Time OCR: Extract text from any content with multi-language support (PaddleOCR / Tesseract)
• Object Segmentation: Precise detection for enhanced content understanding
• Depth Analysis: Foreground/Background separation for immersive focus features
• Pose Detection: Real-time human pose estimation with skeletal overlay
• Super Resolution: AI-powered image enhancement
• Web Overlay Engine: Transform any website with accessibility overlays

Technology Stack

• WebGPU: GPU-accelerated compute shaders for ML inference
• TensorFlow.js: On-device neural networks
• WebGL 2.0: Real-time rendering pipeline
• WASM: Near-native performance
• Privacy-First: All processing happens on-device
• Progressive Web App: Works across all devices

� How to Use

Chrome Extension

1. Build: Run npm run build to generate the production bundle.
2. Install:
   • Open Chrome/Edge and navigate to chrome://extensions.
   • Enable Developer Mode (toggle in top-right).
   • Click Load Unpacked and select the ailly project root folder.
3. Activate: Click the Ailly icon in your toolbar and select "Activate on This Page".
4. Overlay: The Ailly interface will appear as an overlay. Use the floating controller to switch between Webcam (physical world) and Screen (digital content) modes.

Web Application

1. Launch: Run npm run dev and open http://localhost:5173.
2. Settings: Access the settings modal to configure:
   • Quality Profiles:
     • ⚡ Performance: Fastest response, standard resolution (Good for older devices).
     • ⚖️ Balanced: Best mix of speed and quality (Default).
     • 💎 Ultra: Neuro-enhanced resolution and maximum detection accuracy.
   • Presets: Save your custom configuration (e.g., "Reading Mode", "Outdoor") for instant switching.
3. Reference Site: Visit the Feature Reference Page to test accessibility tools like OCR, contrast analysis, and object detection in a controlled environment.

📱 Progressive Web App (PWA)

1. Install:
   • Open http://localhost:5173 (or your deployed URL) in Chrome/Edge.
   • Look for the "Install Ailly" icon in the address bar or the "Install App" button in the navigation header.
   • Follow the prompts to install Ailly as a standalone app.
2. Offline Mode: Ailly caches AI models (up to 32MB) and core assets, allowing it to function completely offline.

Important

WebGPU Requirement: Ailly requires a Secure Context (HTTPS or localhost) to access WebGPU features.

• Localhost: Works out of the box.
• LAN/IP (e.g., 192.168.x.x): Will NOT work unless served over HTTPS. The app will fall back to CPU mode (slower) or fail to initialize 3D features.

🧪 Testing

• Unit Tests: Run npm test to verify core logic (Settings, Services).
• E2E Tests: Run npm run test:e2e to verify user flows (Extension, Settings Persistence).

�🛠️ Development

Prerequisites

• Node.js 18+
• A WebGPU-compatible browser (Chrome 113+, Edge 113+)

Installation

# Install dependencies
npm install

# Start development server
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

Project Structure

ailly/
├── src/
│   ├── core/            # Core Engines (Pipeline, OCR, MediaPipe, Audio)
│   ├── plugins/         # Plugin Architecture (Capture, Vision, UI)
│   ├── ui/              # Visual Components (Overlay, Magnifier)
│   ├── webgpu/          # WebGPU Low-level abstractions
│   ├── shaders/         # WGSL Shader files
│   ├── main.ts          # Application Entry Point
│   └── style.css        # Design System
├── public/              # Static assets
└── REQUIREMENTS.md      # Full technical specification

🎨 Design System

The landing page features a modern, premium design with:

• Vibrant color palette using HSL colors for accessibility
• Smooth animations and micro-interactions
• Glassmorphism effects with backdrop blur
• Responsive typography using clamp() for fluid scaling
• Dark mode optimized with carefully chosen contrast ratios

Color Palette

• Primary: hsl(260, 85%, 65%) - Vibrant purple
• Secondary: hsl(190, 95%, 55%) - Bright cyan
• Accent: hsl(330, 85%, 60%) - Pink accent

📋 Roadmap

See REQUIREMENTS.md for the complete technical specification and product roadmap, including:

1. Foundation Phase: WebGPU setup, basic vision models
2. Core Vision AI: OCR, segmentation, depth estimation
3. Accessibility Layer: Overlay engine, content augmentation
4. Advanced Features: Super-resolution, pose detection
5. Polish & Distribution: Chrome Web Store, performance optimization

🏗️ Architecture

Ailly is built on three foundational pillars:

1. On-Device ML Pipeline: ONNX Runtime Web + WebGPU for efficient model execution
2. Accessibility Overlay Engine: Content injection and real-time augmentation
3. Privacy-First Design: Zero data transmission, all processing local

🤝 Contributing

This project is currently in early development. Contributions, ideas, and feedback are welcome!

📄 License

MIT License - see LICENSE file for details

🌟 Vision

Ailly aims to make the web universally accessible through cutting-edge vision AI technology that runs entirely on your device, ensuring privacy while delivering powerful accessibility features.

---

Built with ❤️ for a more accessible web