Efficient Pipelines for Vision-Based Context Sensing

by   Xiaochen Liu, et al.

Context awareness is an essential part of mobile and ubiquitous computing. Its goal is to unveil situational information about mobile users like locations and activities. The sensed context can enable many services like navigation, AR, and smarting shopping. Such context can be sensed in different ways including visual sensors. There is an emergence of vision sources deployed worldwide. The cameras could be installed on roadside, in-house, and on mobile platforms. This trend provides huge amount of vision data that could be used for context sensing. However, the vision data collection and analytics are still highly manual today. It is hard to deploy cameras at large scale for data collection. Organizing and labeling context from the data are also labor intensive. In recent years, advanced vision algorithms and deep neural networks are used to help analyze vision data. But this approach is limited by data quality, labeling effort, and dependency on hardware resources. In summary, there are three major challenges for today's vision-based context sensing systems: data collection and labeling at large scale, process large data volumes efficiently with limited hardware resources, and extract accurate context out of vision data. The thesis explores the design space that consists of three dimensions: sensing task, sensor types, and task locations. Our prior work explores several points in this design space. We make contributions by (1) developing efficient and scalable solutions for different points in the design space of vision-based sensing tasks; (2) achieving state-of-the-art accuracy in those applications; (3) and developing guidelines for designing such sensing systems.


page 12

page 19

page 24

page 25

page 26

page 29

page 32


OpenRAN Gym: An Open Toolbox for Data Collection and Experimentation with AI in O-RAN

Open Radio Access Network (RAN) architectures will enable interoperabili...

Using Statistical Models to Detect Occupancy in Buildings through Monitoring VOC, CO_2, and other Environmental Factors

Dynamic models of occupancy patterns have shown to be effective in optim...

Vision-Based Proprioceptive Sensing for Soft Inflatable Actuators

This paper presents a vision-based sensing approach for a soft linear ac...

Real-time Scene Segmentation Using a Light Deep Neural Network Architecture for Autonomous Robot Navigation on Construction Sites

Camera-equipped unmanned vehicles (UVs) have received a lot of attention...

Task allocation interface design and personalization in gamified participatory sensing for tourism

The collection of spatiotemporal tourism information is important in sma...

Decoupling Semantic Context and Color Correlation with multi-class cross branch regularization

Success and applicability of Deep Neural Network (DNN) based methods for...

Robotic Inspection and Characterization of Subsurface Defects on Concrete Structures Using Impact Sounding

Impact-sounding (IS) and impact-echo (IE) are well-developed non-destruc...

Please sign up or login with your details

Forgot password? Click here to reset