Bsrgvty

I have seen a plethora of job advertisements in the last few years on mathjobs.org for academic positions in data science. Now I understand why economic pressures would cause this to happen, but from a traditional view of university organization, but how does data science fit in?

I would have guessed that at most, a research group having to do with something labeled "data science" could be formed as an interdisciplinary project between applied mathematicians, statisticians, and computer scientists, with corporate funding. But I don't see why it is a fundamentally distinct intellectual endeavor, prompting mathematics hires specifically in data science.

The first time I heard the term “data science,” it was said that they wanted to take PhD’s who had experience in statistically analyzing large data sets, and train them for a few weeks to apply these skills to marketing and advertising. Now just a few years later people want to hire professors of this.

Question: What about data science is particularly interesting from a mathematical point of view?

I will stay away from the academic politics of hiring "professors of data science", but if I interpret the question more specifically as "does data science offer problems of mathematical interest", I might refer to Bandeira's list of 42 Open Problems in Mathematics of Data Science.

(The full list from 2016 is here, and Bandeira's home page links to solutions of some of these.)

Fundamentally a lot of what a modern data scientist does is very similar to what in previous generations would have been the responsibility of a statistician, and it shouldn't surprise you that there are professors of statistics. Mathematically there are quite a few interesting things that come up in a lot of modern data science, but first let me make a non-comprehensive taxonomy of the sub-areas of data science, because there are several different activities which "data science" includes:

• Data Collection: this is largely a non-mathematical task where data is actually collected. There can be novel mathematical problems solved in this area if one is doing inference because the structure of the collection significantly effects the independence and sampling assumptions of a lot of methods, and that mathematics is usually done in the context of social science or more applied statistics. For example, "Causal Inference without Balance Checking" is a paper about the mathematics of dealing with non-random data collection in inference, written by two economists and a political scientist. The majority of this kind of work is not mathematical, and much more in the realm of computer scientists and social scientists.




• Extraction, Transformation, and Loading (ETL). This is largely the domain of computer scientists, especially whenever you get into issues of "big-data" you are often times talking about running massively parallel algorithms on distributed systems. There is some mathematics that goes into this, even though it is largely not mathematical. For example in the area of Natural Language Processing a key part of this step might be to process words according to a topic model, the most common of which was described in this paper. The underlying model is deeply mathematical, being a baysian generative model, and the paper shows how this work (although done outside of a math department) is mathematical research.




• Inference: This is the domain of classical statisticians, and is all about creating models and estimators from those models to learn something about the population you are sampling from. In the modern practice of data science there are plenty of people who are interested in inference, myself included, and who use the classical tools of statistics to get at it. Interestingly there is an abundance of subjects where the classical tools of inference have been reapproriated to new contexts for prediction. Most interestingly for inference is that there is a lot of new mathematics to be done in taking the new models we are using for prediction and making them usable for inference. For example "Consistency of Random Forests" takes a workhorse of data science and tried to understand its mathematical properties and move towards a place where the otherwise predictive model can be used for inference. Moreover, there is a lot of mathematical work on models utilized by data scientists asking when and how they can be used for an inferential task. The classic example is graphical models, where Judea Pearl's book delves precisely into this question.




• Prediction: This is what the majority of industry data scientists spend the bulk of their time working on. Prediction is often thought about entirely empirically, meaning that very little mathematics goes into it and instead it is based largely on simulation or testing on real data. However, there is math to be done here, both in setting foundations, and in the fact that prediction can be easily re-framed as approximation, a classic topic in analysis. In fact there is a fundamental theorem in machine learning called the Universal Approximation Theorem which is in essence proving a fact about the density and the convex hull of a subspace of $L_2$.

With that groundwork for what data science is out of the way, here are some more specific mathematical issues at play:

• Non Convex Optimization: one of the most common tasks in machine learning is to optimize over some non-convex function. One of the things data scientists wish to understand is the properties of these non-convex optimizations, especially because they are frequently used, but still being understood mathematically. "Non Convex Optimization for Machine Learning" is a monograph that tackles this exact problem, and is very approachable to even the non-mathematician.




• Foundations: I know that when mathematicians think of foundations we often think of the esoteric, but actually in this context what I mean is that, because data science has developed so quickly as an applied discipline, it often discovers that certain models and techniques 'work' but there is quite a bit of mystery about why. For a good introduction into this kind of thinking you can look at a talk like "On the Connection between Neural Networks and Kernels" or a book like Foundations of Data Science by Blum, Hopcraft, and Kannon, which is an undergraduate textbook (so not too advanced) but if you have more training you can easily see some of the deeper issues. So much of data science is deeply rooted in functional analysis, and so I expect to see a lot of work coming from that direction in the future.




• Generative Modeling: This is the problem of approximating a distribution. Clearly there is more traditional work in analysis about interpolation and functional approximation in given functional spaces, and there is also work in probability theory about precisely this problem. In addition to those two traditions, generative modeling also deals a lot with non-parametric estimation. For example the book "Distribution Free Theory of Non-Parametric Regression" is an interesting mathematical take on a lot of methods used classically in non-parametric statistics and by data science in generative modeling.

This is just a sampling of topics, for example I didn't even touch on reinforcement learning, and I think that as time goes on the language and literature surrounding data science will grow into a robust set of literature rooted firmly in analysis and proability theory (with smatterings of geometry and topology).

The Mathematics of Data may go some way towards answering your question. As one example of a mathematically interesting topic that is motivated by data science, you might want to look at the concept of persistent homology.

To begin, there is a family of results which are sometimes referred to as "No free lunch" theorems. Each of these results, in their own way, asserts that any optimization algorithm is just as good as any other if you average over the space of all optimization problems. On the other hand, we know that in specific domains some algorithms vastly outperform all others (that we're aware of) - for detecting objects in images, convolutional neural networks are state of the art, and in computational linguistics the best you can do for most tasks is a neural network with an LSTM or transformer architecture. In both of these cases, the state of the art algorithms perform vastly better than, say, logistic regression.

How can we reconcile the "No free lunch" theorems with our empirical experience? The answer has to be that object detection in images and standard NLP tasks aren't "typical" optimization problems - some combination of the data and the task has some special structure which particular neural architectures are unusually good at detecting. What is this structure? Why are known algorithms so good at learning it? Can we generate new algorithms (neural or otherwise) that are even better?

These are all essentially math problems, sitting somewhere at the intersection of optimization theory and information theory. They are pretty wide open - except in simple cases like logistic regression - there isn't much in the way of theory which characterizes an algorithm as optimal for a particular task among the space of all possible optimization algorithms. An influential paper from 2014 proposes to use the theory of renormalization groups from physics to tackle this question, and there are other attempts using gauge theory or the principle of maximum entropy. Another line of attack involves the so-called "manifold hypothesis", which asserts that real world data sets (presented as a set of points in euclidean space) tend to cluster near a high codimension submanifold.

That's my answer for your main technical question, but I'll also make a remark about the academic politics. There are significantly more job openings in data science than there are people to fill them, so much so that many companies (like AirBnB) have found it cheaper and easier to start internal data science training programs than hire outside people. This problem is not likely to go away any time soon, so it's sensible to incentivize universities to start degree programs in the field, even if it's not yet a fully fleshed-out academic discipline. This has plenty of historical precedent - for instance, academic programs in forensic science and financial mathematics sprouted in the same way in the 1990's.