Dissertations, Theses, and Capstone Projects

Date of Degree

2-2025

Document Type

Thesis

Degree Name

M.S.

Program

Cognitive Neuroscience

Advisor

Edward A. Vessel

Subject Categories

Cognitive Neuroscience | Cognitive Psychology | Cognitive Science | Computational Neuroscience | Quantitative Psychology

Keywords

computational aesthetics, neuroaesthetics, perception, machine vision, machine learning, artificial intelligence

Abstract

Traditional computer vision applications focus on objective tasks, such as object detection, classification, or segmentation. Much of human experience is inherently subjective, such as our personal response to artwork. Individualized experiences are challenging to replicate in a controlled laboratory environment, as those experiences depend on the unique history and internal models of the observer. This work presents a technique to construct a 2-dimensional parameterized stimuli space suitable to induce controlled changes in the perceptual systems of both human and machine observers. Our contributions are threefold:

  1. Using a balanced dataset of 8,800 fine art images spanning 11 art movements and 88 artists from wikiart.org, we embed these images in a 1792-dimensional latent vector space with DreamSim, a generative AI computer vision model optimized to align with human judgments of image similarity. We provide qualitative and quantitative evidence, including t-SNE visualizations, that the embeddings retain sufficient relevant information to meaningfully describe and distinguish various fine art categories.

  2. Applying principal component analysis (PCA), we reduce this space to two principal components that naturally construct a visually addressable metric space with four quadrants. Focusing on an example with two artists we observe that one axis defines a category boundary (separating the two artists), while the other varies with semantic content. Specifically, the upper half of the space features natural imagery like landscapes and trees, while the lower half includes man-made structures such as buildings, ships, streets. This confirms the desired perceptual organization of the space. To evaluate learners’ abilities to generalize across this structured space, we define two types of test sets: an independent-and- identically-distributed (IID) test set and an out-of-distribution (OOD) test set. The IID test set is drawn from the same half of the space as the training data, sharing similar visual and semantic characteristics, whereas the OOD test set consists of images from the opposite half, introducing a distribution shift that challenges learners to classify images with novel characteristics.

  3. We empirically measure the learnability of the space by reporting learning curves, demonstrating that machine learning models can discern the two artists with fewer than 30 pairs of examples, thus setting a lower bound for the dataset size needed for humans to learn the same.

We conclude by sharing preliminary results from an ongoing project to create "personalized" deep neural networks (DNNs) that can serve as proxies for the internal mental representations of humans who have been trained on the same space.

Download

Share

COinS