A bin changes during a shift. Upright parts settle, move, turn face down, or wedge against the wall. Accessible surface orientations therefore vary across the session. A system tuned to a freshly loaded bin degrades as accessibility changes.
This is the canonical partial observability problem in warehouse picking. The robot cannot see the full three dimensional bin state. An overhead depth image shows only the pile's visible surface. Hidden contents and partly obscured parts remain uncertain, so the policy must act from an incomplete view.
Partial Observability at the Controller
The partially observable Markov decision process, or POMDP, formalizes this problem. The robot does not know the true world state, only its current sensor observations. An optimal POMDP policy also uses observation history, which contains information about hidden state that the current view misses.
Keeping the full observation history in a real time control loop is infeasible. The practical approximation is a belief state, a probability distribution over possible world states updated with each observation. Given transition and observation models, it summarizes the history. A good belief state lets the system act without raw history.
The engineering question is how to represent and maintain a belief state on hardware running at production throughput.
Occlusion in Depth Images
A structured light or time of flight depth camera sees the visible surface as a point cloud. Fully occluded parts are invisible. Partly visible parts create incomplete point clouds, making pose estimation ambiguous.
The ambiguity is structured. For a horizontal cylinder, a top down depth image shows the visible arc but reveals little about the end faces or exact axial orientation. For an angled flat part, the depth gradient indicates tilt, but its sign is ambiguous without both edges. Pose estimation must distinguish what the data supports from what it cannot.
The perception module outputs a candidate grasp distribution instead of one pose. Each grasp has a predicted success probability conditioned on the current depth image and the hidden states consistent with it. The policy selects among candidates from this distribution rather than treating the highest confidence candidate as truth.
Lighting Shifts
Warehouse lighting changes through the day. Morning skylight creates fixed hard shadows. Overhead LEDs have different spectral characteristics at operating temperature than at startup. As the facility warms, metal shelving can add thermal reflections in the NIR bands used by many structured light cameras.
A perception model trained at one time or under one lighting condition can vary in accuracy across the shift. Teams generally collect data across the shift cycle. Less common is updating the belief state with lighting as an uncertain variable instead of assuming constant model accuracy.
In an early access deployment, afternoon sunlight at a specific angle caused NIR interference that reduced depth accuracy for a 90 minute window. We adjusted that facility's confidence thresholds for the window. The longer term fix is training across lighting variation. Both responses are needed for production use.
Bin State During Picking
Partial observability also has a temporal dimension. Picks change the bin: parts shift when neighbors are removed, and blocked parts become accessible. The belief state must track both the current view and the expected state after the sequence of picks.
We use a physics informed prior update. After a pick, the policy updates the belief state from the new depth image and a simple forward model of likely shifts among remaining parts. This is not high fidelity physics simulation, but a coarse shift of probability toward settled configurations expected after removal from a location. Though crude, it improves later grasp candidate estimates after partial clearing.
When to Reobserve
A belief state lets the system decide when a better observation is worth acquiring. A second depth scan takes 200 to 500ms, depending on the camera and processing pipeline. In a high throughput cell, the cost matters. Reobserve when expected belief improvement justifies the cycle time.
The policy compares current belief entropy with a threshold calibrated to entropy and grasp success at the facility. Above it, the system queues reobservation; below it, execution proceeds. On an electronics line piloted in 2025, reobservation triggered on about 15% of picks. Those 15% matched high confidence picks in subsequent grasp success. The remaining 85% proceeded directly and performed well because the threshold reflected empirical results.
This Architecture Is Not Universal
Control layer handling helps when uncertainty comes from sensor coverage and part configuration, the dominant warehouse picking case. It does not resolve part identity, which needs different sensing and classification, or task changes. A new untrained SKU can raise belief entropy as expected, but the policy may have no good response.
We built a picking policy for normal variation in accessibility, configuration, and lighting within a stable product set. New products require new demonstrations. That deployment workflow is separate from the partial observability architecture described here. The goal is not certainty. The goal is useful action.