A greedy algorithm is an algorithmic paradigm that builds up a solution piece by piece, always choosing the next piece that offers the most obvious and immediate benefit. This approach makes the choice that seems best at the current moment—the "locally optimal" choice—without regard for future consequences. In machine learning, this is seen in algorithms like decision tree induction (e.g., ID3, C4.5), where at each node, the algorithm greedily selects the feature split that provides the highest information gain at that step, hoping this series of local optima leads to a good overall tree.

A. This describes Bayesian inference or online learning, where a model's parameters are updated sequentially as new data arrives.

B. This describes an exhaustive or brute-force search, which is computationally expensive and the opposite of a greedy strategy's efficiency.

C. This refers to parametric models, a classification of statistical models, not an algorithmic strategy like the greedy approach.

1. Cormen

T. H.

Leiserson

C. E.

Rivest

R. L.

& Stein

C. (2009). Introduction to Algorithms (3rd ed.). MIT Press. In Chapter 16

"Greedy Algorithms

" the introduction states

"A greedy algorithm always makes the choice that looks best at the moment... It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution." (p. 414).

2. Demaine

E. & Devadas

S. (2011). 6.006 Introduction to Algorithms

Fall 2011. Massachusetts Institute of Technology: MIT OpenCourseWare. Lecture 16 notes define the strategy: "A greedy algorithm builds a solution step-by-step

and at each step

it makes a 'locally optimal' choice." (URL: https://ocw.mit.edu/courses/6-006-introduction-to-algorithms-fall-2011/pages/lecture-notes/

Lecture 16: Greedy Algorithms).

3. Roughgarden

T. (n.d.). CS161: Design and Analysis of Algorithms

Lecture #13: Greedy Algorithms. Stanford University. The lecture notes state

"A greedy algorithm is one that

in a series of steps

always makes the choice that is locally optimal." (p. 1).