Blog / · GH-Proxy Team

Laya: Redefining Fast Agent Routing with Non-Autoregressive System 1 Decision Engine

#Open Source #Laya #Decision Engine #AI Inference #GitHub Proxy #Low Latency
Table of Contents6sections

Laya: Redefining Fast Agent Routing with Non-Autoregressive System 1 Decision Engine

In modern AI agent workflows, developers frequently encounter a major latency bottleneck: agents spend hundreds of milliseconds waiting for autoregressive token-by-token generation simply to make a binary decision, route a tool, or evaluate guardrails.

To resolve this inefficiency, Laya emerged as a breakthrough open-source project. Gaining over 25,700+ Stars on GitHub with 21,700+ Stars added in a single week, Laya has surged to the very top of trending rankings.

🚀 Accelerated Access via GH-Proxy

Download releases, clone source code, and fetch model weights at maximum speed using GH-Proxy:

👉 Direct GH-Proxy Acceleration for Laya
(Alternative URL: https://gh-proxy.com/https://github.com/NandhaKishorM/laya)

Fast clone command:

git clone https://gh-proxy.com/https://github.com/NandhaKishorM/laya.git
cd laya
pip install -e .

🧠 Why a Dedicated System 1 Decision Model?

Human cognition relies on two systems:

  1. System 1 (Fast Thinking): Instinctive, sub-second heuristic choices and classification;
  2. System 2 (Slow Thinking): Deliberate, chain-of-thought deductive reasoning.

Large language models like GPT-4, Claude, and Qwen are fundamentally System 2 generators. Laya serves as a purpose-built System 1 engine: bypassing repetitive autoregressive token generation, it delivers structured decisions in a single forward pass.

Three Core Output Primitives

  • Typed Choice: Selects from defined categories or enum options instantly;
  • Continuous Score: Computes normalized confidence and semantic scores (0.0 to 1.0);
  • Yes / No Determinations: Emits deterministic boolean control signals.

✨ Key Technical Highlights

  1. Sub-Millisecond Forward Latency: With fused heads on a distilled encoder backbone, decisions resolve in 5ms to 20ms, an order of magnitude faster than generative LLMs.
  2. 100+ Multilingual Alignment: Out-of-the-box accuracy across English, Chinese, Spanish, and dozens of international languages.
  3. Dynamic Checkpoint Router: Automatically routes incoming queries to the optimal lightweight or specialized checkpoint based on payload complexity.

🛠 Quick Start Example

import laya

engine = laya.DecisionEngine(model="laya-base-multilingual")

# 1. Typed categorical routing in a single pass
route = engine.decide(
    prompt="Route this customer invoice to the corresponding accounting pipeline",
    choices=["finance_ops", "customer_support", "code_generation", "casual_chat"]
)
print(f"Selected route: {route.choice} (Confidence: {route.confidence:.4f})")

# 2. Strict boolean gate
is_critical = engine.boolean(
    statement="Server memory exceeded 95% threshold with IO deadlock detected",
    question="Does this incident warrant an immediate emergency page?"
)
print(f"Trigger alarm: {is_critical}")

🎯 Primary Use Cases

  • Agent Guardrails & Gating: Inspect and route user queries before hitting expensive reasoning models.
  • High-Throughput Ingestion: Process tens of thousands of requests per second for content classification.
  • Edge Deployment: Run deterministic semantic routing locally without external API latency.

💬 Comments & Discussion