Machine Learning Glossary/A

Verfasst von / Written by Sebastian F. Genter

Machine Learning Glossary

A

ablation

Systematic evaluation technique determining component importance through removal analysis. Methodology:

Train baseline model with all components
Remove specific feature/layer/process
Retrain and compare performance

Example: Removing one feature from 10-feature model shows 88% → 55% accuracy drop, indicating critical feature.

A/B testing

Statistical comparison method evaluating two system versions (A=control, B=variant). Key aspects:

Measures performance difference on key metrics (accuracy, CTR)
Determines statistical significance (p-values)
Enables phased rollouts with confidence intervals

accelerator chip

Specialized processors optimizing ML workloads:

Google TPUs: Matrix operations for neural networks
NVIDIA GPUs: Parallel computation architecture

Benefits over CPUs:

10-100x faster training times
Energy-efficient inference
Optimized for tensor operations

accuracy

Basic classification metric: $\frac{Correct Predictions}{Total Predictions}$ Limitations in class imbalance:

99.9% accuracy possible by always predicting majority class
Example: Snow prediction model always saying "no snow"

Use with precision/recall for balanced assessment.

action

Reinforcement learning mechanism enabling state transitions. Components:

Agent selects action via policy
Environment transitions to new state
Reward signal guides learning

Example: Game AI choosing moves to maximize score.

activation function

Neural network nonlinearity introducers:

Function	Formula	Use Case
ReLU	$f (x) = \max (0, x)$	Hidden layers
Sigmoid	$f (x) = \frac{1}{1 + e^{- x}}$	Binary classification

Prevents linear collapse, enables complex pattern learning.

active learning

Strategic data selection approach:

Start with small labeled dataset
Query most informative unlabeled examples
Iteratively expand training set

Applications:

Medical imaging with rare conditions
Legal document analysis

Reduces labeling costs by 30-70% in practice.

AdaGrad

Adaptive gradient algorithm with parameter-specific learning rates:

Accumulates squared gradients per parameter
Automatically adjusts learning pace
Effective for sparse data (NLP features)

Formula: $η_{i} = \frac{η}{\sqrt{G_{i} + ϵ}}$

agent

Autonomous decision-maker in RL systems:

Learns policy mapping states→actions
Types: Model-based vs model-free
Advanced forms: LLM agents using language models for planning

agglomerative clustering

Hierarchical clustering method:

Start with individual data points as clusters
Merge closest cluster pairs repeatedly
Form dendrogram for cluster analysis

Distance metrics: Euclidean, Manhattan, cosine

anomaly detection

Outlier identification techniques:

Statistical: Z-score thresholds
ML-based: Isolation forests, autoencoders

Use cases:

Fraud detection
Network security
Manufacturing defects

AR

(Augmented Reality) Technology blending digital content with real-world environments:

Marker-based: QR code triggers
Markerless: SLAM positioning

Applications: Retail visualization, maintenance guides

area under the PR curve

(PR AUC) Imbalanced classification metric:

X-axis: Recall
Y-axis: Precision
Higher area = better class separation

Preferred over ROC AUC when positive class rare.

area under the ROC curve

(ROC AUC) Binary classification performance measure:

0.5: Random guessing
1.0: Perfect separation
Robust to class distribution changes

Visualizes TPR vs FPR tradeoffs.

artificial general intelligence

(AGI) Hypothetical AI with human-like capabilities:

Cross-domain reasoning
Creative problem solving
Self-improvement capacity

Current status: Not achieved, active research area.

artificial intelligence

Computer systems performing intelligent tasks:

Narrow AI: Specialized systems (chess engines)
General AI: Theoretical human-level intelligence

ML subset focus: Pattern recognition from data.

attention

Neural mechanism weighting input significance:

Scaled Dot-Product: $Attention (Q, K, V) = softmax (\frac{Q K^{T}}{\sqrt{d_{k}}}) V$
Multi-head: Parallel attention layers

Transformers revolutionized NLP through self-attention.

attribute

1. Model input feature (synonym) 2. Protected characteristic in fairness:

Age, gender, ethnicity
Requires bias mitigation strategies

attribute sampling

Decision tree diversification technique:

Random feature subset per node split
Reduces overfitting
Core in Random Forest construction

AUC

(See area under the ROC curve)

augmented reality

(See AR)

autoencoder

Self-supervised architecture components: 1. Encoder: Dimensionality reduction 2. Decoder: Reconstruction attempt Applications:

Anomaly detection
Feature learning
Data denoising

automatic evaluation

Algorithmic model assessment:

Text: BLEU, ROUGE
Code: Unit test pass rates
Images: SSIM, FID scores

Limitations: May miss semantic nuances.

automation bias

Human tendency to over-trust algorithmic outputs:

Medical diagnosis support systems
Autonomous vehicle overrides

Mitigation: Confidence displays, human verification steps.

AutoML

Automated Machine Learning components:

Neural architecture search
Hyperparameter optimization
Feature engineering

Tools: Google Cloud AutoML, H2O Driverless AI

autorater evaluation

Hybrid assessment system:

Train ML model on human ratings
Automate scoring while preserving rater patterns
Continuous feedback loop

Balances scalability with judgment quality.

auto-regressive model

Sequential generation approach:

Text: GPT series
Images: PixelCNN, Parti

Characteristic: Next token prediction based on prior outputs

auxiliary loss

Secondary training objectives:

Guide early layer learning
Combat vanishing gradients

Example: Intermediate layer predictions in Inception networks

average precision at k

Information retrieval metric: Fehler beim Parsen (Syntaxfehler): {\displaystyle \text{AP}@k = \frac{1}{|\text{Relevant}|} \sum_{i=1}^k P(i) \cdot \text{rel}(i)} Where $P (i)$ is precision at position $i$

axis-aligned condition

Decision tree splits using single features:

"Income > $50k"
"Age ≤ 30"

Contrast with oblique splits combining multiple features