Dissertations and Theses
Date of Award
2025
Document Type
Thesis
Department
Computer Science
First Advisor
Zhigang Zhu
Keywords
Virtual Environments, Camera Calibration, Sensor Network
Abstract
Crowded places are increasingly targets of violence due to the increased accessibility and covertness of weapons, explosives, and other technology like drones. Addressing the challenges of protecting crowded places and assisting vulnerable individuals requires a multidisciplinary approach, taking inspiration from many different perspectives. Video surveillance of these crowded public facilities, such as train and bus stations, airports, shopping malls, and sports arenas, is very important to public safety, both for identifying threats/terrorist attacks and implementing evacuation plans.
The work of this thesis is part of a larger project aiming to explore the potential of using real-time computer vision and deep learning algorithms with the Boston Dynamics robotic dog Spot for the modeling of large public venues, and in its collaboration and interaction with 3D model of the large public venue, a network of surveillance cameras monitoring the area, and humans in the environment. The thesis work focuses on the exploration of the following two parts: (1) Calibrating Multiple Camera Feeds Using the Mobile Robotic Dog Spot; (2) Building 3D Live Virtual Environments in the Unity3D Game Engine.
The goal is to calibrate the network of cameras monitoring the area by using the walking Spot to generate known 3D calibration target points accurate enough and widely distributed across the field of view of a camera, and estimate the intrinsic and extrinsic parameters of the cameras. The current progress includes creating a 3D virtual environment of a robotics lab where virtual cameras take snapshots and collect locations of Spot moving through the space, which are then used to calibrate the virtual cameras.
This work seeks to address the ever-increasing security concerns in public venues and has significant scientific and societal impacts, including enhancing research in digital twins, smart surveillance, robotics, and assistive technology. Those in charge of security and protecting the public need to be supported by the latest technological advances to stay ahead of the threats. Overall, the goal of the work presented here is to make the duties of those who are tasked with protecting public places more efficient in detecting, deterring, and mitigating targeted violence.
Recommended Citation
Samoylov, Eltan, "Virtual Environment Creation and Camera Calibration for Soft Target Identification and Assistance in Crowded Spaces with a Sensor Network and a Robotic Dog" (2025). CUNY Academic Works.
https://academicworks.cuny.edu/cc_etds_theses/1263
Comments
This thesis was based on the work developed under an appointment to the DHS Summer Research Team Program (SRT) for Minority Serving Institutions, and the Follow-On Project, administered for the U.S. Department of Homeland Security (DHS) by the Oak Ridge Institute for Science and Education (ORISE) through an interagency agreement between DHS and the U.S. Department of Energy (DOE). ORISE is managed by Oak Ridge Associated Universities (ORAU) under DOE contract number DE-SC0014664. The thesis has not been formally reviewed by DHS.
The author of this thesis, Eltan Samoylov, would like to sincerely thank the project team, especially the thesis advisor, Professor Zhigang Zhu at the City College of New York, and SRT hosts Professors Jie Gong and Fred Roberts at Rutgers, the State University of New Jersey, for their advising and guidance.
The thesis is part of the work reported in an overview article published in the SPIE Proceedings 2025 - Paper Number 13458-15, titled “Real-time computer vision and deep learning for 3D environment modeling, camera network calibration and human-robot interaction using a robot dog” with authors Zhigang Zhu, Jie Gong, Chong Di, Eltan Samoylov, Brandon Vasquez, Haiqiao Liu, Shengyuan Feng, and Fred Roberts. More details are provided in the thesis for the work of the thesis advisor, who would like to thank all the collaborators listed in the SPIE paper.
The views and conclusions contained in this document were based on those of the author and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DHS, DOE, or ORAU/ORISE. DHS, DOE, and ORAU/ORISE do not endorse any products or commercial services mentioned in this thesis.