Machine Learning Glossary/D

Verfasst von / Written by Sebastian F. Genter

---

• Machine Learning Glossary

data analysis

Process of inspecting, cleansing, and modeling data to discover patterns and support decision-making. Techniques include:

• Descriptive statistics
• Data visualization (histograms, scatter plots)
• Correlation analysis

Critical first step before model development.

data augmentation

Artificial dataset expansion through transformations:

• Images: Rotation, flipping, color adjustment
• Text: Synonym replacement, back-translation
• Audio: Pitch shifting, noise injection

Improves model robustness and generalization.

DataFrame

Pandas data structure for tabular data manipulation:

• Columnar storage with labeled axes
• Supports heterogeneous data types
• Enables SQL-like operations (groupby, merge)

data parallelism

Distributed training strategy:

• Replicates model across devices
• Splits batch across replicas
• Synchronizes gradients

Enables large-batch training on GPU/TPU clusters.

Dataset API (tf.data)

TensorFlow pipeline construction toolkit:

• Efficient data loading
• Preprocessing transformations
• Iterator-based access

Supports both in-memory and streaming data.

data set or dataset

Collection of structured records for ML:

• Formats: CSV, Parquet, TFRecord
• Splits: Train/validation/test
• Versioning crucial for reproducibility.

decision boundary

Hypersurface separating different classes:

• Linear: Straight line/plane
• Nonlinear: Complex curves

Visualizable in 2D/3D feature spaces.

decision forest

Ensemble of decision trees:

• Random Forest: Bagging with feature sampling
• Gradient Boosted Trees: Sequential error correction

Reduces variance through collective prediction.

decision threshold

Probability cutoff for class assignment:

• Default 0.5 for binary classification
• Tuned via ROC curve analysis

Impacts false positive/negative rates.

decision tree

Rule-based model with hierarchical conditions:

• Nodes: Feature tests
• Leaves: Final predictions

Interpretable but prone to overfitting.

decoder

Network component generating outputs:

• Seq2seq: Produces target sequence from encoder state
• Autoencoder: Reconstructs input from latent space
• Transformer: Generates tokens autoregressively.

deep model

Neural network with multiple hidden layers:

• Typically >3 hidden layers
• Learns hierarchical feature representations

Requires large datasets for effective training.

deep neural network

(See deep model)

Deep Q-Network (DQN)

Reinforcement learning architecture combining:

• Q-learning value estimation
• Neural network function approximation

Key innovations: Experience replay, target networks.

demographic parity

Fairness criterion requiring:

• Equal positive prediction rates across groups
• $P(\hat{Y}=1|A=a)=P(\hat{Y}=1|A=b)$

Controversial when base rates differ between groups.

denoising

Self-supervised pretraining strategy:

1. Corrupt inputs (mask tokens, add noise)
2. Train model to reconstruct originals

Builds robust representations from unlabeled data.

dense feature

Numerical features with mostly non-zero values:

• Temperature readings
• Pixel intensities

Stored as floating-point arrays (contrast sparse features).

dense layer

Fully connected neural network layer:

• Each neuron connects to all inputs
• Computes $output=activation(Wx+b)$

Common in final classification stages.

depth

Neural network complexity measure:

• Hidden layers + output layer
• Excludes input layer

Modern architectures may have 100+ layers.

depthwise separable convolutional neural network (sepCNN)

Efficient CNN variant separating:

1. Depthwise convolution (spatial filtering)
2. Pointwise convolution (channel mixing)

Reduces parameters while maintaining performance.

derived label

Proxy target when direct labels are unavailable:

• User clicks → content relevance
• Purchase history → product preference

Requires careful validation of label quality.

device

Hardware execution context:

• CPU: General-purpose processing
• GPU/TPU: Accelerated matrix operations

Managed via frameworks like TensorFlow Device API.

differential privacy

Data protection framework guaranteeing:

• Individual contributions indistinguishable
• Formal privacy budget (ε)

Implemented through noise addition and clipping.

dimension reduction

Techniques compressing feature space:

• Linear: PCA, LDA
• Nonlinear: t-SNE, UMAP

Preserves important patterns while reducing noise.

dimensions

1. Tensor rank:
   • Scalar (0D), Vector (1D), Matrix (2D)
2. Feature vector length
3. Embedding space size

direct prompting

Zero-shot learning approach:

• No examples provided
• Relies on model's pretrained knowledge

Example: "Translate 'Hello' to French:"

discrete feature

Categorical variables with finite values:

• Product categories
• Zip codes

Requires encoding (one-hot, embeddings) for model use.

discriminative model

Learns conditional probability $P(Y|X)$:

• Focuses on class boundaries
• Examples: Logistic regression, CRFs

Contrasts with generative models.

discriminator

GAN component distinguishing real/fake data:

• Trained to maximize detection accuracy
• Guides generator improvement through adversarial loss.

disparate impact

Unintended discriminatory effects:

• Neutral policies affecting groups differently
• Measured via 80% rule: $\frac{P(positive|minority)}{P(positive|majority)}\ge 0.8$

disparate treatment

Explicit use of protected attributes:

• Direct discrimination in decision rules
• Illegal in many jurisdictions (credit, housing).

distillation

Knowledge transfer technique:

• Trains compact student model
• Mimics outputs/logits of large teacher model

Enables deployment on resource-constrained devices.

distribution

Statistical characterization of data:

• Normal (Gaussian)
• Power law
• Multimodal

Understanding distributions guides preprocessing choices.

divisive clustering

Top-down hierarchical approach:

1. Start with all data in one cluster
2. Recursively split clusters

Produces dendrogram showing split hierarchy.

downsampling

Class imbalance mitigation:

• Reduces majority class samples
• Often combined with minority oversampling

Risk: Loses potentially important majority examples.

DQN

(See Deep Q-Network)

dropout regularization

Training technique deactivating random neurons:

• Prevents co-adaptation
• Acts as implicit ensemble

Common rate: 0.2-0.5.

dynamic

Real-time adaptation systems:

• Online learning: Continuous model updates
• Dynamic inference: Adjusts computation per input

Contrasts with static batch processing.

dynamic model

Continuously updated system:

• Adapts to concept drift
• Examples: Recommendation systems, fraud detection

Requires robust monitoring pipelines.