The Early Stopping Problem Where Bayesian Optimization Cannot Be Applied
Recommended Post: 【Philosophy】 Philosophy Table of Contents
When to stop. When is the optimal choice.
We ponder when to stop exploring and make a decision in important life decisions. This issue can be explained through the Bayesian Optimization approach. In general, Bayesian optimization is based on Gaussian processes and assumes that the objective function follows a Gaussian distribution to find the optimal choice. However, the real world is an uncertain and chaotic system, so this assumption may not hold. Additionally, lacking domain knowledge about the real world makes it challenging to apply Bayesian optimization with other assumed distributions. Then, how should the Early Stopping problem be approached in the real world?
The following practical strategies were diagnosed:
Strategy 1. Utilizing instincts, intuition, attraction, and impulses
○ Don’t just judge with your head; use intuitive information you feel with your whole body.
○ If it’s a choice you truly like and are proud of, even if you fail, you can get back up and eventually succeed.
Strategy 2. Not fearing failure and trying again
○ Maintain an attitude of finding ways to try again instead of feeling discouraged.
○ The attitude of “buying more time to try more” is important and connects with Strategy 4.
○ When it’s tough, will you give up? Or, will you be relentless? - Jeff Bezos
Strategy 3. Not limiting choices to a binary of 0 and 1
○ Not all choices are as clear-cut as black and white.
○ Understanding these intermediate choices helps promote flexible thinking. However, be cautious not to make unethical choices.
Strategy 4. Admissions are luck, but applications are a choice
○ While outcomes cannot be controlled, how much effort we put into trying is our choice. To increase opportunities, one must actively strive.
Strategy 5. Embracing and enjoying uncertainty and instability
○ Being impatient can lead to failure even when success was possible.
○ It’s important to maintain a flexible and relaxed attitude in uncertain situations.
Strategy 6. Assigning lower weights to confusing problems from the start
○ Life is full of countless problems. Adjusting importance levels yourself and spending less energy on confusing issues is one approach.
Input: 2025.01.04 18:00