#### Date of Degree

6-2017

#### Document Type

Dissertation

#### Degree Name

Ph.D.

#### Program

Computer Science

#### Advisor

Delaram Kahrobaei

#### Committee Members

Benjamin Fine

Robert Haralick

Delaram Kahrobaei

Vladimir Shpilrain

Xiaowen Zhang

#### Subject Categories

Algebra | Artificial Intelligence and Robotics | Discrete Mathematics and Combinatorics | Theory and Algorithms

#### Keywords

group theory, machine learning, non-commutative cryptography, conjugacy, polycyclic group

#### Abstract

Machine learning and pattern recognition techniques have been successfully applied to algorithmic problems in free groups. In this dissertation, we seek to extend these techniques to finitely presented non-free groups, in particular to polycyclic and metabelian groups that are of interest to non-commutative cryptography.

As a prototypical example, we utilize supervised learning methods to construct classifiers that can solve the conjugacy decision problem, i.e., determine whether or not a pair of elements from a specified group are conjugate. The accuracies of classifiers created using decision trees, random forests, and *N*-tuple neural network models are evaluated for several non-free groups. The very high accuracy of these classifiers suggests an underlying mathematical relationship with respect to conjugacy in the tested groups.

In addition to testing these techniques on several well-known finitely presented groups, we introduce a new family of metabelian groups for which we analyze the computational complexity of the conjugacy search problem. We prove that for the family in general the time complexity of the conjugacy search problem is exponential, while for a subfamily the problem is polynomial. We also show that for some of these groups the conjugacy search problem is an instance of the discrete logarithm problem.

We also apply machine learning techniques to solving the conjugacy search problem. For each platform group we train a *N*-tuple regression network that can produce a candidate conjugator for a pair of conjugate elements. This candidate is then used as the initial state of a local search for a conjugator in the Cayley graph, in what we call regression-based conjugacy search (RBCS). RBCS can be applied to groups such as polycyclic groups for which other heuristic approaches, such as the length-based attack, are ineffective.

#### Recommended Citation

Gryak, Jonathan, "Solving Algorithmic Problems in Finitely Presented Groups via Machine Learning" (2017). *CUNY Academic Works.*

https://academicworks.cuny.edu/gc_etds/2045

#### Included in

Algebra Commons, Artificial Intelligence and Robotics Commons, Discrete Mathematics and Combinatorics Commons, Theory and Algorithms Commons