Research Signal – Source date 2022-11-20
Give a computer measurements and ask it for an equation. A useful answer might reveal how the measured quantities relate, in a form a scientist can inspect. But the computer must choose among mathematical possibilities, and several expressions may describe the available points almost equally well.
Symbolic regression is the task of learning mathematical expressions from data. In the November 20, 2022 preprint, the authors survey approaches to this task and include small experiments that expose a surprising difficulty. Giving a method more kinds of mathematical functions can make recovering the original equation harder. A highly accurate fit can also come from an expression different from the one that generated the data.
The appeal is easy to understand through the authors’ illustration of electric charges, their separation and the force between them. Suppose a researcher chooses a formula that adds together weighted contributions from each charge and the distance. Fitting that formula means estimating its coefficients—the numbers that determine each contribution’s size. The basic form has already been decided.
Symbolic regression opens up that earlier decision. What mathematical relationship should connect the inputs? A compact equation lets a researcher trace how changing an input changes the output. That makes the answer inspectable, although readability alone does not establish a causal mechanism.
The mathematical menu shapes the search
An equation-search method needs a set of permitted ingredients. In one approach examined in the survey, those ingredients are collected in a function library. The method then fits a combination of them to the data.
The authors compare two libraries. The smaller one contains powers of the input, from x through x⁹. It can represent polynomials by assigning coefficients to those terms. The expanded library retains those powers and adds sine, cosine, tangent, exponential and sigmoid functions.
This changes the range of relationships available to the method. A polynomial library restricts the search to combinations of powers. Adding other functions permits different shapes, but also introduces more alternatives that the measurements must help distinguish.
The survey places this regression-based approach alongside methods based on expression trees and approaches inspired by physics or mathematics. An expression tree represents how the parts of a formula fit together. The broader task can therefore involve selecting and arranging mathematical operations, as well as estimating numerical coefficients.
A larger library misses a known polynomial
For their illustrative experiments, the authors generate synthetic data from known functions. Each experiment has twenty training points and twenty test points. The input values in both sets are sampled uniformly between −1 and 1, and the chosen function supplies the corresponding outputs.
Knowing the generating function makes two separate checks possible. The researchers can ask how well the returned expression matches the outputs, and whether it recovers the expression used to create them.
Consider the example called Nguyen-2, whose generating function is:
x⁴ + x³ + x² + x
With the polynomial library, the method returns the exact function and achieves an R² of 1.0. With the expanded library, it does not recover the generating expression, and the reported R² is 0.886. R² measures how closely predictions match the variation in the outputs; 1.0 represents a perfect fit on the evaluated data.
The original polynomial remains available in the larger library. Adding sine, cosine and the other functions has not removed the correct ingredients. Yet the displayed result becomes worse. The authors explain that different functions can behave very similarly over the covered range, making it difficult to distinguish them using the sampled points. In their discussion of polynomial examples, they also note that the quality of the expanded-library fit depends on the training dataset and improves with more training data.
The distinction between fitting and recovering becomes especially clear in two mixed-function examples. Using the polynomial library, the method achieves R² values of 0.999 for functions involving sine and cosine, while returning approximate rather than exact expressions. A score very close to 1 can therefore coexist with a different mathematical answer. The score describes agreement with the evaluated outputs; it does not certify that the original expression has been found.
These examples show how the choice of library interacts with a small dataset. They do not establish that larger libraries always perform worse. Here, the researchers know the answer in advance and can identify exact recovery directly. That makes the exercise informative about the method’s behavior, without turning recovery of a synthetic function into discovery of a previously unknown physical law.
Searching becomes a computational problem too
Ambiguity among similar-looking functions is only one difficulty. A method must also find useful expressions efficiently as the number of inputs and possible combinations grows.
The survey describes genetic programming, a longstanding approach that evolves a population of candidate expressions through biology-inspired operations. Instead of starting with a single fixed formula and adjusting its coefficients, this search works through changing candidate forms over successive iterations.
The survey identifies difficulties in scaling genetic-programming approaches to datasets with many input variables, along with sensitivity to hyperparameters—the settings that govern how a method runs. The practical challenge includes both exploring the possible expressions and choosing settings that make the exploration effective.
The authors also describe growing interest in neural approaches to symbolic regression. Some methods combine neural networks with reinforcement learning. This broadens the set of tools available for equation searches, but the survey’s discussion here does not establish a winner between neural and evolutionary approaches. Computational efficiency is a separate question from whether the measurements can distinguish the candidate relationships.
What the comparisons actually test
The twenty-point test sets provide inputs separate from those used for fitting, but they come from the same interval as the training inputs. The demonstration therefore tests performance on further samples within that range. It does not establish how the returned equations behave beyond −1 to 1.
That distinction matters because the authors describe symbolic regression as serving two ambitions. It can supply an interpretable predictive model, and it can support scientific discovery by proposing relationships that generalize to conditions not yet covered by data. Testing new points within a familiar interval addresses a narrower question than testing a new regime.
The survey separately describes SRBench, an external benchmarking effort covering fourteen symbolic-regression methods across 252 datasets; ten of the methods are based on genetic programming. Its breadth differs substantially from the authors’ small worked examples. Those coverage figures describe the scope of a comparison effort. By themselves, they provide neither a method ranking nor evidence that an equation will work outside its sampled conditions.
What this could change
For researchers studying physical systems, the potential payoff is a route from measurements to compact mathematical hypotheses. If equation-search methods can recover useful relationships, handle more complex inputs and produce expressions that survive tests beyond the sampled conditions, those expressions could help scientists reason about regimes they have not yet measured.
A researcher could inspect how an inferred relationship depends on distance, charge or another physical quantity, then work out what it predicts as conditions change. Such predictions could help direct experiments toward unexplored behavior and connect separate measurements through a common mathematical description. The scientific capability at stake is the ability to turn an observed pattern into a relationship that supports further reasoning and testable predictions.
Key Sources
- Primary source: Interpretable Scientific Discovery with Symbolic Regression: A Review
arXiv · Preprint · 2022-11-20
Keep exploring