A 10ms control period forces local Neuralzome policy inference. Industrial controllers run at 100Hz to 500Hz, so the outer task loop must issue an action within each period: 10ms at 100Hz and 8ms at 125Hz. Even a tuned AWS connection from a typical factory in India takes 40-80ms round-trip. Cloud execution cannot meet the requirement.
Local execution was the baseline constraint from day one. The runtime architecture follows from it.
Where the 50ms Budget Goes
Four stages carry data from sensor observation to actuator command. Their allocations explain the architecture.
Sensor acquisition and pre-processing: 5-8ms. Raw data from cameras and force/torque sensors arrives over USB 3 or Ethernet, depending on the sensor. Pre-processing includes depth image denoising, point cloud generation, and F/T bias removal. We run this on the host CPU rather than the inference accelerator to avoid round-trip overhead, and we pipeline it so the next sensor acquisition starts while the previous frame is still being processed.
State estimation and fusion: 4-7ms. The raw sensor data passes into the state estimator, which maintains the running world model and produces the structured observation vector that the policy inference step consumes. The estimator runs on CPU. Its computational load is dominated by the pose estimation step for object localization, which we run as a lightweight point-cloud matching algorithm rather than a deep-network detector to keep latency predictable.
Policy inference: 20-35ms. This is the dominant step. The policy network takes the observation vector and outputs a probability distribution over actions, from which we sample the action for the current cycle. We run this on a dedicated inference accelerator (typically an NVIDIA Jetson Orin NX or equivalent, depending on the customer's hardware). The 20-35ms range reflects the range of policy sizes we deploy: simpler policies for single-arm pick-and-place sit toward the lower bound; more complex policies for multi-step assembly with contact reasoning sit toward the upper bound.
Constraint checking and command generation: 3-5ms. The proposed action passes through the constraint satisfaction layer (collision avoidance, workspace limits, force limits), and the final joint-space command is generated and dispatched to the robot controller. This runs on CPU and is deterministically bounded.
Worst case is 8 + 7 + 35 + 5 = 55ms. To target 50ms, the common path must be faster, with headroom for variance at every stage.
Hardware Platform Choices
We choose edge platforms with a dedicated neural processing unit (NPU) or tensor core cluster. Memory bandwidth and small-batch inference latency matter more than peak FLOPS. Benchmarks often use batch size 32 or 64, while robot control uses batch size 1: one observation and one action per cycle. High throughput with poor single-sample latency is the wrong fit.
We characterize accelerator candidates by measuring p99 latency at batch size 1 for our largest deployed policy network. The p99 metric matters more than mean latency because the control loop is time-critical and occasional spikes above the budget cause real problems. A platform with mean latency 25ms but p99 latency 80ms is not fit for purpose regardless of its mean performance.
Sustained-load thermal behavior is the second accelerator check. Some edge accelerators throttle during continuous inference because they target bursts. Robot loops run through eight-hour shifts, so 25ms for the first 20 minutes followed by 60ms after thermal steady state is unacceptable.
Model Quantization and Its Limits
Quantizing the policy network from FP32 to INT8 or FP16 cuts latency and memory use, but robot control demands care. A calibrated INT8 policy usually matches FP32 on the nominal task. The concern is distribution shift: novel object geometry, unusual part orientations, and atypical contact patterns that training data covered sparsely.
On out-of-distribution observations, FP32 policies produce lower-confidence, more conservative action distributions. Quantized INT8 policies can be more confident but less calibrated because reduced precision smooths the output. Overconfidence on unfamiliar inputs is more dangerous than signaling uncertainty and pausing.
We deploy FP16 rather than INT8. On platforms with native FP16 tensor support, it cuts latency by roughly 30% versus FP32 while retaining more calibration under distribution shift. A systematic calibration-preserving scheme remains in development.
Pipeline vs. Synchronous Execution Tradeoff
Pipelining the four stages can improve throughput: while the accelerator runs policy inference for observation N, the estimator handles N+1 and sensor acquisition captures N+2. The command then uses an observation one or two cycles old.
That lag suits scenes changing slowly. In contact-rich work, force state can change within 5-10ms, such as contact onset during a press-fit. A stale observation may delay response by one or two cycles, adding 2-4mm at 200mm/s.
We pipeline sensor acquisition and state estimation continuously, but policy inference waits for the freshest state estimate. It may wait a few milliseconds, trading latency for fresher input. Contact tasks take this trade; relatively static pick-and-place tasks use full pipelining for throughput.
Network Communication: What Stays Local
Policy weights, the state estimator, constraint layer, and robot command interface run on the edge node. The real-time path uses no network during operation. Network functions are policy updates between shifts, telemetry logging, and the management API for configuration, monitoring, and remote diagnostics.
This boundary is deliberate. We considered cloud fallback when confidence fell below a threshold, using a larger model for uncertain observations. Analysis rejected it because variable cloud round-trips cannot fit a deterministic budget, especially when contact uncertainty makes latency critical. Improving local inference was the better path.
The architectural discipline this imposes is straightforward: if a capability cannot be delivered within 50ms locally, it does not go in the real-time control path. If it is genuinely needed, it becomes a background process that enriches the state estimate on longer timescales rather than trying to fit into the control loop directly.