Dissertations, Theses, and Capstone Projects

Date of Degree

9-2025

Document Type

Doctoral Dissertation

Degree Name

Doctor of Philosophy

Program

Computer Science

Advisor

Michael Mandel

Committee Members

Johanna Devaney

Elena Filatova

Mark Cartwright

Subject Categories

Other Computer Engineering

Keywords

Low-resource, AI, Ecoacoustic, Audio, LLM, machine learning

Abstract

Ecoacoustic monitoring via machine learning enables scalable analysis but is often constrained by labeled data scarcity, particularly in remote regions like the Arctic. This thesis confronts low-resource ecoacoustic audio classification by developing and evaluating complementary machine learning methodologies. We introduce EDANSA, the first publicly available, expert- labeled Arctic dataset of its kind, curated via novel active learning, alongside a baseline CNN. We systematically evaluate transfer learning, showing general audio embeddings effectively bootstrap classifiers for challenging Arctic sounds, significantly outperforming direct label mapping. Optimizing label utility, we investigate standard data augmentation and introduce novel audio data valuation via Shapley values, revealing strategies to maximize performance and identify annotation inconsistencies, while finding self-supervised pre-training less impactful without extensive adaptation. Broadening the data sources, we develop a cross-modal approach integrating satellite weather and audio for high-temporal-resolution rain/wind prediction without manual labels. Finally, exploring advanced AI, we critically evaluate Multimodal Large Language Models (MLLMs) using mechanistic interpretability, revealing fundamental limitations in audio-text reasoning. Collectively, these contributions offer validated methods, unique datasets, and critical insights for robust ecoacoustic monitoring and broader challenges in low-resource machine learning, cross-modal integration, and AI reasoning.

Download

Share

COinS