What is an estimator?
In machine learning, an estimator is an equation for picking the “best,” or most likely accurate, data model based upon observations in realty. Not to be confused with estimation in general, the estimator is the formula that evaluates a given quantity (the estimand) and generates an estimate. This estimate is then inserted into the deep learning classifier system to determine what action to take.
Uses of Estimators
By quantifying guesses, estimators are how machine learning in theory is implemented in practice. Without the ability to estimate the parameters of a dataset (such as the layers in a neural network or the bandwidth in a kernel), there would be no way for an AI system to “learn.”
A simple example of estimators and estimation in practice is the so-called “German Tank Problem” from World War Two. The Allies had no way to know for sure how many tanks the Germans were building every month. By counting the serial numbers of captured or destroyed tanks (the estimand), Allied statisticians created an estimator rule. This equation calculated the maximum possible number of tanks based upon the sequential serial numbers, and apply minimum variance analysis to generate the most likely estimate for how many new tanks German was building.
Types of Estimators
Estimators come in two broad categories—point and interval. Point equations generate single value results, such as standard deviation, that can be plugged into a deep learning algorithm’s classifier functions. Interval equations generate a range of likely values, such as a confidence interval, for analysis.
In addition, each estimator rule can be tailored to generate different types of estimates:
- Biased - Either an overestimate or an underestimate.
- Efficient - Smallest variance analysis. The smallest possible variance is referred to as the “best” estimate.
- Invariant: Less flexible estimates that aren’t easily changed by data transformations.
- Shrinkage: An unprocessed estimate that’s combined with other variables to create complex estimates.
- Sufficient: Estimating the total population’s parameter from a limited dataset.
- Unbiased: An exact-match estimate value that neither underestimates nor overestimates.
