Abstract: In recent times, computational efficiency has witnessed a tremendous development, making it a vital element in many sectors such as applied sciences, medicine, industry, technology, and mathematics in general. However, these techniques face significant challenges related to the ability to apply them effectively and safely. Although there is much research on numerical optimization methods in mathematics, there is a lack of directions that combine these studies and clarify the common trends between them. This review aims to provide a comprehensive analysis of recent studies related to the applications of numerical optimization methods in solving unrestricted optimization problems and nonlinear equations of different shapes and classifications, specifically the derivative-free optimization (DFO) method, focusing on the challenges facing researchers in applying and dealing with them, the opportunities available for the future, and ways to expand them. Derivative-Free Optimization (DFO) refers to a set of optimization techniques used when derivatives of the objective function are not available or are unreliable. This can occur in various practical scenarios, such as when dealing with noisy measurements, complex simulations, or black-box functions where the mathematical form of the objective is not explicitly known.
Keyword: Derivative-Free Optimization Methods, optimization problems, Direct numerical methods.
