Verfasst von / Written by Sebastian F. Genter
Q
Q-function
- reinforcementLearning
In reinforcement learning, the Q-function (also known as the state-action value function) is a fundamental function that evaluates the expected utility or value of taking a specific action in a particular state. The function, often denoted as $Q(s, a)$, is defined as the expected cumulative future reward that an agent can expect to receive by taking action $a$ in state $s$, and then following an optimal policy thereafter. Reinforcement learning algorithms, such as Q-learning, aim to learn this Q-function to enable the agent to make optimal decisions in its environment.
Q-learning
- reinforcementLearning
Q-learning is a popular, off-policy reinforcement learning algorithm. Its primary goal is to learn the optimal Q-function, $Q(s, a)$, which represents the maximum expected future reward achievable by taking action $a$ in state $s$. The algorithm iteratively updates its estimate of the Q-function based on the rewards received and the estimated future rewards from subsequent states. Q-learning is an off-policy method because it learns the value of the optimal policy independently of the policy the agent is currently following to explore the environment.
quantile
- fundamentals
A quantile is a value that divides a probability distribution or a set of ordered data into equal-sized, contiguous subpopulations. For a dataset, quantiles mark the points that divide the data into intervals, with each interval containing a specified proportion of the data points. Commonly used quantiles include quartiles (dividing the data into four equal parts), deciles (ten equal parts), and percentiles (one hundred equal parts). Quantiles are useful in data analysis for understanding the distribution of data, identifying potential outliers, and are used in techniques like quantile bucketing.
quantile bucketing
- fundamentals
Quantile bucketing is a feature engineering technique used to convert numerical data into categorical data by dividing the range of the numerical feature into bins based on quantiles. Instead of creating bins of equal width (as in equal-width bucketing), quantile bucketing ensures that each bin contains approximately the same number of data points. The boundaries of the bins are determined by the calculated quantiles of the data. This technique is particularly useful when the data is skewed or has a non-uniform distribution, as it helps to create more balanced bins compared to equal-width bucketing.
quantization
- fundamentals
Quantization is the process of mapping a continuous range of input values to a smaller, finite set of discrete output values. In machine learning, quantization typically refers to reducing the precision of the numbers used to represent model parameters and activations, often converting them from floating-point representations (e.g., 32-bit or 16-bit) to lower-precision integer representations (e.g., 8-bit). This technique is commonly used during model compression to reduce the model's size, memory footprint, and computational cost, making it more suitable for deployment on resource-constrained devices.
queue
- TensorFlow
In TensorFlow, a queue is an operation that implements a standard queue data structure. Queues are used in TensorFlow to manage the flow of data into the training or inference process, particularly when dealing with asynchronous data loading or processing pipelines. Data can be enqueued (added) to the queue and dequeued (removed) from the queue in a First-In, First-Out (FIFO) manner. Queues were more prevalent in older versions of TensorFlow (like 1.x) for building input pipelines and handling asynchronous operations.