Dissertations, Theses, and Capstone Projects

Date of Degree

9-2025

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy

Program

Computer Science

Advisor

Saptarshi Debroy

Committee Members

Sarah Ita Levitan

Lena Mashayekhy

Kaliappa Ravindran

Subject Categories

Computer Sciences

Keywords

Edge Computing, Reinforcement Learning, Edge-AI, Distributed Systems

Abstract

Modern end-user applications that are highly compute- and data-intensive, while being extremely latency- and accuracy-sensitive, are increasingly reliant on distributed computing. This paradigm spans a range of architectures, from cloud computing to in-device processing. Cloud computing, though scalable, often incurs high latency and cost, constraints that are particularly problematic for time-sensitive applications. In contrast, in-device computing on end or IoT devices is limited by resource constraints, making it inadequate for many complex workloads. Edge computing presents a compelling alternative by bringing computation closer to data sources, thereby reducing end-to-end latency and improving responsiveness. However, the inherent decentralized and dynamic nature of many edge environments, often relying on limited capacity and/or volunteer resources, introduces unique challenges in the way of satisfying the underlying applications’ performance and security requirements. Traditional approaches to modeling and optimization depend on strong assumptions and complete system knowledge, which are rarely available in such unpredictable settings. Reinforcement learning (RL), by contrast, offers a solution that can adapt to uncertainty and learn effective behavior directly from interaction, making it especially well-suited for these environments.

This thesis employs RL towards developing frameworks for edge resource management, designed to address system unpredictability, instability, component heterogeneity, and lack of prior system knowledge that is inherent to dynamic and resource-constrained edge environments supporting data- and resource-intensive applications. These frameworks are customized to enable adaptive decision-making across a range of real-world scenarios and applications: First, we design an RL-driven scheduling framework for data-intensive scientific workflows in volunteer edge-cloud (VEC) environment. Our approach jointly considers workflow requirements, node preferences, and diverse resource policies to support robust task allocation. Second, we design RL-based {\em Infer-EDGE} framework for optimizing deep neural network (DNN) inference in “Just-in-Time” edge deployments. Built on the Advantage Actor-Critic (A2C) algorithm, Infer-EDGE dynamically balances trade-offs between latency, accuracy, and energy consumption for latency-sensitive image/video processing tasks in loosely coupled, resource-constrained environments. Finally, we develop an RL-based framework for reliable resource management for latency-sensitive multi-view 3D reconstruction, primarily adopted for mission-critical surveillance and reconnaissance applications. In such highly dynamic environments, where image quality, network performance, and resource availability fluctuate unpredictably, our framework learns to make effective camera and edge server selection decisions. Comprehensive evaluations, across both simulations and testbed deployments, show that the proposed RL-based frameworks consistently enhance reliability, reconstruction quality, and end-to-end performance, learning effective behavior purely from feedback signals without requiring prior knowledge of the environment. Overall, this thesis illustrates how RL can be leveraged to support robust, adaptive behavior in edge systems where traditional, static decision-making approaches fall short.

Download

Share

COinS