I Built a 100x Faster Android Automation Tool Because AI Agents Deserve Better

Every AI agent that touches an Android device hits the same wall. Not permissions. Not compatibility. Latency.

A single tap takes 300–1000ms through conventional tools. A UI tree dump? 1–5 seconds through ADB. Fill a 40-field form and you're staring at your screen for 17 minutes while your "intelligent" agent crawls through a protocol stack designed for human debuggers, not AI agents.

I've spent years building Android tools — from SmartScreen (which anticipated Google's Adaptive Timeout by nine years) to production apps serving thousands of users. When I started building AI agents that needed to control Android devices, I expected the tooling to be ready. It wasn't even close.

So I built NeuralBridge — an open-source Android companion app that gives AI agents sub-10ms control over any Android device. No ADB process spawning. No five-hop protocol chains. No middleware servers.

Just one HTTP call from agent to device.

⚡ The Problem Nobody's Talking About

Here's what happens when an AI agent taps a button through a conventional tool:

Agent → HTTP Request → Middleware Server → ADB Bridge → UIAutomator2 → Device

Five hops. Each one adds latency, error surface, and complexity. The "fast" path through other MCP-based tools isn't much better — every action spawns a new ADB process (100–200ms overhead) plus a UIAutomator dump (1–5 seconds).

This isn't a minor inconvenience. It's an architectural bottleneck that makes real-time AI automation impossible.

Consider a realistic scenario: an AI agent filling out a health insurance form. 40 fields. Each tap-and-type cycle takes ~500ms through conventional tools. That's 20 seconds of pure waiting — and that's the optimistic case. With mandatory settle delays, you're looking at minutes.

The agent isn't thinking. It isn't reasoning. It's waiting for infrastructure to catch up.

🏗️ One Hop Instead of Five

NeuralBridge's key insight is deceptively simple: put the MCP server inside the Android app itself.

Agent → HTTP → NeuralBridge Companion App (in-process execution)

One hop. That's it.

The companion app embeds a Ktor CIO HTTP server on port 7474, speaking native MCP protocol. When an agent sends a tap command, it doesn't cross process boundaries, spawn ADB shells, or traverse middleware. The app's AccessibilityService executes the gesture in-process, directly through Android's dispatchGesture() API.

The result: a tap completes in ~2ms — down from the 300–500ms that conventional tools require.

My rule for this project was simple: if you're crossing a process boundary to do something the OS can do in-process, you've already lost. Every millisecond of IPC overhead is a tax on intelligence. I'd rather spend six months putting the server inside the app than accept a 300ms floor on every action.

Why AccessibilityService?

Most Android automation tools treat ADB as the primary interface. ADB is powerful, but it was designed for debugging — not for real-time control by AI agents. Every ADB command:

• Spawns a new OS process
• Establishes a USB/TCP connection
• Serializes the command
• Waits for the Android runtime to process it
• Deserializes the response

AccessibilityService bypasses all of this. It runs inside the Android runtime with direct access to the view hierarchy, gesture dispatch, and screen content. No IPC. No serialization. No process spawning.

NeuralBridge uses a two-tier strategy that routes ~95% of operations through this fast path:

The slow path exists because Android intentionally restricts certain operations from non-system apps. But even at 200ms, it's still faster than most tools' fast path.

📊 The Numbers That Matter

I ran benchmarks against every major Android automation tool. Here's what I found:

That's 100x faster on average. NeuralBridge numbers measured on a Pixel 7 running Android 14 over WiFi, averaged across 100 runs per operation. Competitor numbers sourced from their official documentation and community benchmarks (linked in the full benchmark doc). I'd welcome head-to-head benchmarks from the community — if you run your own, open an issue with results.

The form-filling scenario tells the story better than averages:

• NeuralBridge: 40 fields × ~100ms (tap + type + navigate) = ~4 seconds
• Conventional tools: 40 fields × ~25 seconds (with settle delays) = ~17 minutes

Same task. Same device. Same AI agent. The gap widens further when you factor in mandatory settle delays between actions.

To be fair about what these numbers don't show: Some existing tools have decade-old ecosystems with drivers for iOS, Windows, and web — NeuralBridge is Android-only. Some tools use intentional settle delays (750ms+) that buy near-zero flakiness at the cost of speed. And ADB-forwarded TCP approaches work over USB without needing WiFi connectivity. NeuralBridge wins on raw latency and token efficiency, but it's not a drop-in replacement for every use case.

🧠 Designed for AI, Not Adapted for AI

Most automation tools were built for human-driven test scripts and later retrofitted with MCP adapters. NeuralBridge was built AI-native from day one. Three design decisions make this tangible:

1. Semantic Selectors (Not Pixel Coordinates)

AI agents think in terms of "tap the Login button," not "tap pixel (540, 820)." NeuralBridge's selector system bridges this gap with a six-step resolution chain:

1. Exact text match
2. Partial text match (contains)
3. Content description match
4. Resource ID match
5. Combined AND match (multiple criteria)
6. Fuzzy match (tolerance for typos and variations)

When multiple elements match, a tiebreaker picks the most likely target: visible elements beat hidden ones, interactive elements beat static ones, center-positioned elements beat edge ones.

The agent says tap(text="Login"). NeuralBridge handles the rest.

2. Token-Optimized Responses

Every token an AI agent processes costs money and time. A typical 200-node Android screen generates ~3,000 tokens of UI tree data in conventional tools. NeuralBridge compresses this to ~800 tokens — a 73% reduction — without losing actionable information.

How:

• Compact format: Tabular representation instead of verbose JSON
• Interactive-only filter: Strips ~80% of non-interactive decoration nodes
• Smart omission: Empty fields, null values, and redundant metadata are dropped
• Compressed bounds: [l,t,r,b] instead of {"left":0,"top":0,"right":1080,"bottom":2400}

This optimization is transparent — agents don't need to request it. It's the default. Your AI agent spends less on API calls and more on actual reasoning.

3. Context-Rich Observations

The get_screen_context tool returns both a screenshot and the semantic UI tree in a single call (~70ms). This gives agents what human testers get intuitively: a visual snapshot paired with actionable element metadata. One call instead of two. Half the latency. Complete situational awareness.

🔧 32 Tools That Show AI-Native Design

NeuralBridge ships with 32 tools organized across seven categories: observe, act, manage, wait, test, device, and meta. Rather than listing them all, here are three that show how the AI-native design plays out in practice:

get_screen_context — Returns a screenshot and semantic UI tree in a single ~70ms call. Most tools force agents to make two round-trips (screenshot + tree dump), doubling latency and token cost. One call, complete situational awareness.

scroll_to_element — The agent says "find the Submit button." NeuralBridge scrolls through the page, checking the accessibility tree after each scroll, until it finds the element or gives up after 30 seconds. No pixel-guessing. No "scroll down 3 times and hope."

accessibility_audit — Checks every element on screen for touch target size, missing content descriptions, and contrast issues. Returns results in under 50ms. What used to require a manual pass with Android's Accessibility Scanner now happens as a single MCP call.

The full tool reference covers all 32.

🔬 The Technical Details That Took Months

Building NeuralBridge wasn't just about the architecture insight. The hard part was making it production-grade across Android's fragmented ecosystem.

Screenshot Pipeline

Screenshots use MediaProjection with a JNI bridge to libjpeg-turbo for JPEG encoding in C++. The result: ~60ms for a full-resolution capture. When MediaProjection consent resets (Android 14+ does this on app restart), NeuralBridge automatically falls back to ADB screencap — slower, but reliable.

Screenshots are only useful if the agent can reliably refer back to the elements it sees.

Stable Element IDs

Android's built-in view IDs are unstable — they change across app updates, language switches, and even screen rotations. NeuralBridge generates hash-based element IDs from the element's structural position and content. Same element, same ID, regardless of what changes around it.

Cross-Version Compatibility

The app runs on Android 7.0 through Android 15+. That's nine major versions with different permission models, accessibility APIs, and security restrictions. Each version required specific handling:

• Android 10+: Background clipboard access restricted → ADB fallback
• Android 14+: MediaProjection consent lifecycle changes → automatic re-consent handling
• Android 15+: Restricted settings for accessibility → manual enablement required

Cold Start Reality

There's one number I'm not proud of: the first request after app launch takes 30–120 seconds. Ktor's CIO engine JIT-compiles on the first incoming connection, and there's no way to avoid it short of switching to a pre-compiled server. Every subsequent request is sub-10ms, but that first one hurts. I'm exploring AOT compilation options to kill this, but for now, it's a known tradeoff — slow start, fast everything after.

🌍 Why This Matters Now

Three things happened in the last 12 months: Claude gained computer-use capabilities, Google launched Project Mariner, and OpenAI shipped Operator. The industry has decided that AI agents will control real software on real devices. But the infrastructure is still stuck in the ADB era.

The gap between "AI can reason about what to do on a phone" and "AI can actually do it fast enough to be useful" is enormous. Right now, an agent can plan a 10-step workflow in 200ms and then spend 30 seconds executing it. NeuralBridge closes that gap.

🚧 What NeuralBridge Can't Do

Games that render via OpenGL, Unity, or Unreal have no accessibility tree — NeuralBridge can't see or interact with their UI elements. Banking apps with FLAG_SECURE block screenshots. Biometric prompts can't be automated. And Google Play won't distribute apps that use AccessibilityService for automation, so NeuralBridge is sideloaded.

NeuralBridge is built for standard Android apps — which covers ~95% of what AI agents need to interact with.

When an agent can execute 40 UI actions in 4 seconds instead of 17 minutes, new categories of automation become possible:

• Real-time app testing where the AI adapts its strategy based on what it sees, without waiting seconds between observations
• Multi-app workflows — Gmail to Calendar to Maps — completed in the time it takes a human to unlock their phone
• Accessibility auditing at scale — automatically checking touch targets, content descriptions, and contrast ratios across every screen in an app
• Continuous integration for mobile — running full UI test suites in seconds, not hours

🚀 Getting Started

NeuralBridge is open source (Apache 2.0) and requires no root access.

Setup:

1. Install the companion APK on your Android device
2. Enable the NeuralBridge AccessibilityService in Settings
3. Grant screen capture permission on first use
4. Point your AI agent to the device's IP:

{
  "mcpServers": {
    "neuralbridge": {
      "type": "http",
      "url": "http://<device-wifi-ip>:7474/mcp"
    }
  }
}

That's it. Your agent now has sub-10ms control over any Android device on your network. No middleware server. No ADB forwarding. No middleware.

Security note: The MCP server currently runs without authentication — anyone on your WiFi network can send commands. This is fine for development and testing on trusted networks, but don't expose it on public WiFi. Token-based auth is on the roadmap.

The Bigger Picture

I didn't build NeuralBridge because the world needed another automation framework. I built it because I was tired of watching AI agents — systems capable of sophisticated reasoning — bottlenecked by infrastructure that treats latency as an afterthought.

The first generation of mobile automation tools was built for human testers writing scripts. The second generation bolted on AI adapters. NeuralBridge is what happens when you start from zero with a single question: what would mobile automation look like if it were designed for AI agents from the ground up?

The answer is 100x faster, 73% cheaper on tokens, and fits in a single Android app.

What's next: Multi-device orchestration, WebView inspection tools, and iOS support. If you want to see how fast your agents can actually move, grab the code and build APK. If you find a tool that's missing, open an issue — I read every one.

NeuralBridge is open source (Apache 2.0) on GitHub.

I build the tools I wish existed — and then open-source them. Follow me on HackerNoon for more on AI agents, Android internals, and the infrastructure nobody else wants to write.