Search papers, labs, and topics across Lattice.
The paper introduces EgoSelf, a system for personalized egocentric assistance that leverages a graph-based interaction memory to capture user-specific behaviors and contexts. This memory encodes temporal and semantic relationships between interaction events and entities, allowing for the derivation of user profiles. A personalized learning task is then formulated to predict future interactions based on individual user histories stored in the graph, demonstrating the system's effectiveness.
Forget generic assistants – EgoSelf learns your habits from your first-person view data to predict your future interactions.
Egocentric assistants often rely on first-person view data to capture user behavior and context for personalized services. Since different users exhibit distinct habits, preferences, and routines, such personalization is essential for truly effective assistance. However, effectively integrating long-term user data for personalization remains a key challenge. To address this, we introduce EgoSelf, a system that includes a graph-based interaction memory constructed from past observations and a dedicated learning task for personalization. The memory captures temporal and semantic relationships among interaction events and entities, from which user-specific profiles are derived. The personalized learning task is formulated as a prediction problem where the model predicts possible future interactions from individual user's historical behavior recorded in the graph. Extensive experiments demonstrate the effectiveness of EgoSelf as a personalized egocentric assistant. Code is available at \href{https://abie-e.github.io/egoself_project/}{https://abie-e.github.io/egoself\_project/}.
