Stochastic Optimization Methods and Their Applications in Machine Learning
Date:
- Introduced the family of stochastic optimization algorithms and their motivation over classical gradient-based methods for non-convex, high-dimensional search spaces.
- Covered Simulated Annealing — probabilistic hill-climbing inspired by the annealing process in metallurgy, used for combinatorial optimization and hyperparameter search.
- Explained Genetic Algorithms (GA) — population-based evolutionary search using selection, crossover, and mutation operators, applied to feature selection and neural architecture search.
- Presented Ant Colony Optimization (ACO) — swarm intelligence algorithm modeled on foraging behavior, used for routing, scheduling, and graph-based ML problems.
- Detailed the Black Hole Algorithm — a nature-inspired metaheuristic where candidate solutions orbit a best solution (the black hole) and are absorbed if they cross the event horizon, applied to feature subset selection.
- Demonstrated comparative performance of these methods on feature selection benchmarks, showing improvements in model accuracy and dimensionality reduction over filter-based baselines.