Towards Low-Power and Internet-Scale Mixed Reality

Low power consumption and scalability are the key properties of augmented reality (AR) yet to be realized to attract more users and drive new applications. Towards low-power single-user AR, we develop a novel framework, MARLIN, which only uses a DNN as needed, to detect new objects or recapture objects that significantly change in appearance. When multiple AR users collaborate in a common space, we propose FreeAR that employs collaborative time slicing to distribute compute-heavy AR tasks such as SLAM and DNNs across devices, and leverages P2P communications for message exchanges, to achieve a low energy footprint without edge infrastructure. With cellular infrastructure, AR devices can communicate with cloud servers to offload heavy computations. However, we discover that when multiple users communicate over cellular networks, AR applications experience high latencies. We characterize and understand the root causes of high latencies and perform a first-of-its-kind measurement study. Based on the study, we propose network-aware and network-agnostic AR solutions that help ramp up AR goodput by ~40-70%.

Co-sponsored by: New Jersey Institute of Technology

Speaker(s): Dr. Kittipat Apicharttrisorn,

Room: The Lewis P. and Julia Kiernan Conference Room, Bldg: Electrical and Computer Engineering Center Suite 200, New Jersey Institute of Technology, University Heights, Newark, New Jersey, United States, 07102, Virtual: https://events.vtools.ieee.org/m/349644