---
source: arxiv
url: http://arxiv.org/abs/2604.07392v1
published_at: '2026-04-08T06:14:46'
authors:
- Fan Zhaowen
topics:
- world-model
- memory-augmented-retrieval
- uav-navigation
- interpretable-control
- embodied-decision-making
relevance_score: 0.84
run_id: materialize-outputs
language_code: en
---

# Event-Centric World Modeling with Memory-Augmented Retrieval for Embodied Decision-Making

## Summary
This paper proposes an event-centric world model for embodied decision-making that stores past situations and retrieves similar ones to choose actions. The system is aimed at dynamic, safety-critical control, with a UAV navigation study used as the main validation.

## Problem
- Autonomous agents in dynamic environments need decisions that are fast, physically consistent, and easy to inspect.
- End-to-end policies can map observations to actions, but the paper argues that they hide why an action was chosen and make it hard to check physical feasibility.
- This matters in safety-critical settings such as UAV flight, where inconsistent or opaque actions can lead to collisions or unstable behavior.

## Approach
- The environment is converted into a set of semantic events: detected objects, their positions and motion, global context, the agent state, and goal direction.
- A permutation-invariant encoder maps that event set into a latent code, so the agent can compare the current situation with stored past situations.
- A knowledge bank stores triples of latent event code, maneuver, and reward or feasibility score; action selection uses nearest-neighbor retrieval and similarity weights over the top-k matches.
- To avoid averaging incompatible actions, retrieved maneuvers are clustered by direction, and the final action is averaged only inside the highest-weight cluster.
- The latent dynamics are constrained to be contractive and physics-aware, using a stable linear transition model, Lyapunov-style bounds, a physics-consistency regularizer, and ANN retrieval with FAISS for real-time use.

## Results
- In NVIDIA Isaac Sim UAV experiments, the expert dataset used for pretraining contains **27,075** collision-free trajectories generated by a Virtual Potential Field supervisor.
- The control loop is designed for **20-50 ms** update intervals, and the paper claims **sub-millisecond** retrieval latency on embedded hardware using FAISS-based ANN search.
- In an adversarial curriculum with **5 episodes**, the training loss reportedly spikes to about **77.9** in Episode 2 and then drops to about **2.6** by Episode 5.
- In the reported navigation evaluation, the agent achieves **100% success rate** with **0 collisions** across all five curriculum episodes.
- The average trajectory length is about **680 steps per 100 m** task.
- The excerpt does not provide baseline comparisons against other methods, ablations, or standard benchmark metrics beyond these internal results.

## Link
- [http://arxiv.org/abs/2604.07392v1](http://arxiv.org/abs/2604.07392v1)