In this paper the procedures to test global search algorithms applied to space trajectory design problems are discussed. Furthermore, a number of performance indexes that can be used to evaluate the effectiveness of the tested algorithms are presented. The performance indexes are then compared and the actual significance of each one of them is highlighted. Three global optimization algorithms are tested on three typical space trajectory design problems

Penalty methods are a well known class of algorithms for constrained optimization. They transform a constrained problem into a sequence of unconstrained \emph{penalized} problems in the hope that approximate solutions of the latter converge to a solution of the former. If Lagrange multipliers exist, exact penalty methods ensure that the penalty parameter only need increase a finite number of times, but are typically scorned in smooth optimization for the penalized problems are not smooth. This led researchers to consider the implementation of exact penalty methods inconvenient. Recent advances in proximal methods have led to increasingly efficient solvers for nonsmooth optimization. We study a general exact penalty algorithm and use it to show that the exact $\ell_2$-penalty method for equality-constrained optimization can, in fact, be implemented efficiently by solving the penalized problem using a proximal-type algorithm. We study the convergence of our algorithm and establish a worst-case complexity bound of $\mathcal{O}(ε^{-2})$ to bring a stationarity measure below $ε> 0$ under the Mangarasian-Fromowitz constraint qualification and Lipschitz continuity of the objective gradient and constraint Jacobian. While the Lipschitz continuity of the objective gradient is not required for convergence in view of recent works, it is used in our analysis to derive the complexity bound. In a degenerate scenario where the penalty parameter grows unbounded, the complexity becomes $\mathcal{O}(ε^{-8})$, which is worse than another bound found in the literature. Finally, we report numerical experience on small-scale problems from a standard collection and compare our solver with an augmented-Lagrangian and an SQP method. Our preliminary implementation is superior to the augmented Lagrangian in terms of robustness and efficiency, and is competitive with the SQP method.

In this paper we discuss the procedures to test a global search algorithm applied to a space trajectory design problem. Then, we present some performance indexes that can be used to evaluate the effectiveness of global optimization algorithms. The performance indexes are then compared highlighting the actual significance of each one of them. A number of global optimization algorithms are tested on four typical space trajectory design problems. From the results of the proposed testing procedure we infer for each pair algorithm-problem the relation between the heuristics implemented in the solution algorithm and the main characteristics of the problem under investigation. From this analysis we derive a novel interpretation of some evolutionary heuristics, based on dynamical system theory and we significantly improve the performance of one of the tested algorithms

Combinatorial optimization is widely applied in a number of areas nowadays. Unfortunately, many combinatorial optimization problems are NP-hard which usually means that they are unsolvable in practice. However, it is often unnecessary to have an exact solution. In this case one may use heuristic approach to obtain a near-optimal solution in some reasonable time. We focus on two combinatorial optimization problems, namely the Generalized Traveling Salesman Problem and the Multidimensional Assignment Problem. The first problem is an important generalization of the Traveling Salesman Problem; the second one is a generalization of the Assignment Problem for an arbitrary number of dimensions. Both problems are NP-hard and have hosts of applications. In this work, we discuss different aspects of heuristics design and evaluation. A broad spectrum of related subjects, covered in this research, includes test bed generation and analysis, implementation and performance issues, local search neighborhoods and efficient exploration algorithms, metaheuristics design and population sizing in memetic algorithm. The most important results are obtained in the areas of local search and memetic algorithms for the considered problems. In both cases we have significantly advanced the existing knowledge on the local search neighborhoods and algorithms by systematizing and improving the previous results. We have proposed a number of efficient heuristics which dominate the existing algorithms in a wide range of time/quality requirements. Several new approaches, introduced in our memetic algorithms, make them the state-of-the-art metaheuristics for the corresponding problems. Population sizing is one of the most promising among these approaches; it is expected to be applicable to virtually any memetic algorithm.

This paper describes the basic ideas of several commonly used intelligent optimization algorithms, summarizes their essential features, classifies them, and points out the improvement directions.

Niching methods extend genetic algorithms and permit the investigation of multiple optimal solutions in the search space. In this paper, we review and discuss various strategies of niching for optimization in electromagnetics. Traditional mathematical problems and an electromagnetic benchmark are solved using niching genetic algorithms to show their interest in real world optimization

This paper surveys the primary computational hurdles of Energy Systems optimization coming from different sources: model-induced complexity, optimization algorithm requirements, and uncertainties handling (both aleatoric and epistemic). Techniques to reduce complexity such as time-series and spatial aggregation, model order reduction, and specialized optimization strategies are reviewed for their effectiveness in balancing computational feasibility and model fidelity. Furthermore, Various uncertainty-management frameworks, including scenario-based approaches, robust optimization, and distributionally robust methods, are reviewed and their limitations in scaling and data requirements are discussed. The potential of hybrid modeling emerges as a key avenue: by fusing mechanistic and machine learning elements, hybrid techniques for modelling and optimization can harness the strengths of both worlds while mitigating their respective drawbacks. The paper highlights several directions for further research to develop advanced methods to tackle the complexity of MES.

Copyright @ 2006 ACMIn this paper, a new gene based adaptive mutation scheme is proposed for genetic algorithms (GAs), where the information on gene based fitness statistics and on gene based allele distribution statistics are correlated to explicitly adapt the mutation probability for each gene locus over time. A convergence control mechanism is combined with the proposed mutation scheme to maintain sufficient diversity in the population. Experiments are carried out to compare the proposed mutation scheme to traditional mutation and two advanced adaptive mutation schemes on a set of optimization problems. The experimental results show that the proposed mutation scheme e ciently improves GA's performance

Bayesian optimization has emerged at the forefront of expensive black-box optimization due to its data efficiency. Recent years have witnessed a proliferation of studies on the development of new Bayesian optimization algorithms and their applications. Hence, this paper attempts to provide a comprehensive and updated survey of recent advances in Bayesian optimization and identify interesting open problems. We categorize the existing work on Bayesian optimization into nine main groups according to the motivations and focus of the proposed algorithms. For each category, we present the main advances with respect to the construction of surrogate models and adaptation of the acquisition functions. Finally, we discuss the open questions and suggest promising future research directions, in particular with regard to heterogeneity, privacy preservation, and fairness in distributed and federated optimization systems.

This article presents a comprehensive overview of advanced decision-making methodologies employed in autonomous vehicle navigation. It examines the intricate interplay between perception systems, AI-driven algorithms, and path-planning mechanisms that enable self-driving cars to navigate complex environments safely and efficiently. The article discusses the role of multisensor fusion in creating accurate environmental maps, the application of machine learning and neural networks in real-time decision-making, and the challenges of path planning and execution in dynamic scenarios. Furthermore, it addresses current limitations and future research directions, including the need for improved handling of unpredictable human behavior, adaptation to diverse road conditions, and ethical considerations in autonomous driving. By synthesizing recent advancements and ongoing challenges, this article provides valuable insights into the cutting-edge technologies shaping the future of transportation.

The solution of the transportation network optimization problem actually requires, in most cases, very intricate and powerful computer resources, so that it is not feasible to use classical algorithms. One promising way is to use stochastic search techniques. In this context, Genetic Algorithms (GAs) seem to be - among all the available methodologies- one of the most efficient methods able to approach transport network design and optimization. Particularly, this paper will focus the attention on the possibility of modelling and optimizing Public Bus Networks by means of GAs. In the proposed algorithm, the specific class of Cumulative GAs(CGAs) will be used for solving the first level of the network optimization problem, while a classical assignment model ,or alternatively a neural network approach ,will be adopted for the Fitness Function(FF) evaluation. CGAs will then be utilized in order to generate new populations of networks, which will be evaluated by means of a suitable software package. For each new solution some indicators will be calculated .A unique FF will be finally evaluated by means of a multicriteria method. Altough the research is still in a preliminary stage, the emerging first results concerning numerical cases show very good perspectives for this new approach. A test in real cases will also follow.

This review explores the application of intelligent optimization algorithms to Multi-Objective Optimal Power Flow (MOPF) in enhancing modern power systems. It delves into the challenges posed by the integration of renewables, smart grids, and increasing energy demands, focusing on evolutionary algorithms, swarm intelligence, and deep reinforcement learning. The effectiveness, scalability, and application of these algorithms are analyzed, with findings suggesting that algorithm selection is contingent on the specific MOPF problem at hand, and hybrid approaches offer significant promise. The importance of standard test systems for verifying solutions and the role of software tools in facilitating analysis are emphasized. Future research is directed towards exploiting machine learning for dynamic optimization, embracing decentralized energy systems, and adapting to evolving policy frameworks to improve power system efficiency and sustainability. This review aims to advance MOPF research by highlighting state-of-the-art methodologies and encouraging the development of innovative solutions for future energy challenges.

This is a post-print version of the article - Copyright @ 2008 Springer-VerlagEvolutionary computation has become an important problem solving methodology among the set of search and optimization techniques. Recently, more and more different evolutionary techniques have been developed, especially hybrid evolutionary algorithms. This paper proposes an island based hybrid evolutionary algorithm (IHEA) for optimization, which is based on Particle swarm optimization (PSO), Fast Evolutionary Programming (FEP), and Estimation of Distribution Algorithm (EDA). Within IHEA, an island model is designed to cooperatively search for the global optima in search space. By combining the strengths of the three component algorithms, IHEA greatly improves the optimization performance of the three basic algorithms. Experimental results demonstrate that IHEA outperforms all the three component algorithms on the test problems.This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/1

Optimizing the mRNA codon has an essential impact on gene expression for a specific target protein. It is an NP-hard problem; thus, exact solutions to such optimization problems become computationally intractable for realistic problem sizes on both classical and quantum computers. However, approximate solutions via heuristics can substantially impact the application they enable. Quantum approximate optimization is an alternative computation paradigm promising for tackling such problems. Recently, there has been some research in quantum algorithms for bioinformatics, specifically for mRNA codon optimization. This research presents a denser way to encode codons for implementing mRNA codon optimization via the variational quantum eigensolver algorithms on a gate-based quantum computer. This reduces the qubit requirement by half compared to the existing quantum approach, thus allowing longer sequences to be executed on existing quantum processors. The performance of the proposed algorithm is evaluated by comparing its results to exact solutions, showing well-matching results.

Feature spaces optimization plays a very important role in object recognition and categorization. After analyzing of several fashionable local features at present, some optimization algorithms based on the information theory are proposed. In this paper, we describe the approaches to recognize generic objects using these features which have been optimized. As baselines for comparison, we also implemented some additional recognition systems using other optimization algorithms. The performance analysis on the obtained experimental results demonstrates that the proposed optimization algorithms are effective and efficient.

Copyright @ Springer-Verlag 2010.This paper proposes the use of the q-Gaussian mutation with self-adaptation of the shape of the mutation distribution in evolutionary algorithms. The shape of the q-Gaussian mutation distribution is controlled by a real parameter q. In the proposed method, the real parameter q of the q-Gaussian mutation is encoded in the chromosome of individuals and hence is allowed to evolve during the evolutionary process. In order to test the new mutation operator, evolution strategy and evolutionary programming algorithms with self-adapted q-Gaussian mutation generated from anisotropic and isotropic distributions are presented. The theoretical analysis of the q-Gaussian mutation is also provided. In the experimental study, the q-Gaussian mutation is compared to Gaussian and Cauchy mutations in the optimization of a set of test functions. Experimental results show the efficiency of the proposed method of self-adapting the mutation distribution in evolutionary algorithms.This work was supported in part by FAPESP and CNPq in Brazil and in part by the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant EP/E060722/1 and Grant EP/E060722/2

Federated Learning (FL) has emerged as a promising technique for edge devices to collaboratively learn a shared machine learning model while keeping training data locally on the device, thereby removing the need to store and access the full data in the cloud. However, FL is difficult to implement, test and deploy in practice considering heterogeneity in common edge device settings, making it fundamentally hard for researchers to efficiently prototype and test their optimization algorithms. In this work, our aim is to alleviate this problem by introducing FL_PyTorch : a suite of open-source software written in python that builds on top of one the most popular research Deep Learning (DL) framework PyTorch. We built FL_PyTorch as a research simulator for FL to enable fast development, prototyping and experimenting with new and existing FL optimization algorithms. Our system supports abstractions that provide researchers with a sufficient level of flexibility to experiment with existing and novel approaches to advance the state-of-the-art. Furthermore, FL_PyTorch is a simple to use console system, allows to run several clients simultaneously using local CPUs or GPU(s), and even remote compute devices without the need for any distributed implementation provided by the user. FL_PyTorch also offers a Graphical User Interface. For new methods, researchers only provide the centralized implementation of their algorithm. To showcase the possibilities and usefulness of our system, we experiment with several well-known state-of-the-art FL algorithms and a few of the most common FL datasets.

Copyright @ Springer-Verlag 2005.Evolutionary algorithms have been widely used for stationary optimization problems. However, the environments of real world problems are often dynamic. This seriously challenges traditional evolutionary algorithms. In this paper, the application of population-based incremental learning (PBIL) algorithms, a class of evolutionary algorithms, for dynamic problems is investigated. Inspired by the complementarity mechanism in nature a Dual PBIL is proposed, which operates on two probability vectors that are dual to each other with respect to the central point in the genotype space. A diversity maintaining technique of combining the central probability vector into PBIL is also proposed to improve PBILs adaptability in dynamic environments. In this paper, a new dynamic problem generator that can create required dynamics from any binary-encoded stationary problem is also formalized. Using this generator, a series of dynamic problems were systematically constructed from several benchmark stationary problems and an experimental study was carried out to compare the performance of several PBIL algorithms and two variants of standard genetic algorithm. Based on the experimental results, we carried out algorithm performance analysis regarding the weakness and strength of studied PBIL algorithms and identified several potential improvements to PBIL for dynamic optimization problems.This work was was supported by UK EPSRC under Grant GR/S79718/01

Optimal Transport (OT) has established itself as a robust framework for quantifying differences between distributions, with applications that span fields such as machine learning, data science, and computer vision. This paper offers a detailed examination of the OT problem, beginning with its theoretical foundations, including the classical formulations of Monge and Kantorovich and their extensions to modern computational techniques. It explores cutting-edge algorithms, including Sinkhorn iterations, primal-dual strategies, and reduction-based approaches, emphasizing their efficiency and scalability in addressing high-dimensional problems. The paper also highlights emerging trends, such as integrating OT into machine learning frameworks, the development of novel problem variants, and ongoing theoretical advancements. Applications of OT are presented across a range of domains, with particular attention to its innovative application in time series data analysis via Optimal Transport Warping (OTW), a robust alternative to methods like Dynamic Time Warping. Despite the significant progress made, challenges related to scalability, robustness, and ethical considerations remain, necessitating further research. The paper underscores OT's potential to bridge theoretical depth and practical utility, fostering impactful advancements across diverse disciplines.

Copyright @ Springer-Verlag 2008Dynamic optimization problems challenge traditional evolutionary algorithms seriously since they, once converged, cannot adapt quickly to environmental changes. This paper investigates the application of memetic algorithms, a class of hybrid evolutionary algorithms, for dynamic optimization problems. An adaptive hill climbing method is proposed as the local search technique in the framework of memetic algorithms, which combines the features of greedy crossover-based hill climbing and steepest mutation-based hill climbing. In order to address the convergence problem, two diversity maintaining methods, called adaptive dual mapping and triggered random immigrants, respectively, are also introduced into the proposed memetic algorithm for dynamic optimization problems. Based on a series of dynamic problems generated from several stationary benchmark problems, experiments are carried out to investigate the performance of the proposed memetic algorithm in comparison with some peer evolutionary algorithms. The experimental results show the efficiency of the proposed memetic algorithm in dynamic environments.This work was supported by the National Nature Science Foundation of China (NSFC) under Grant Nos. 70431003 and 70671020, the National Innovation Research Community Science Foundation of China under Grant No. 60521003, and the National Support Plan of China under Grant No. 2006BAH02A09 and the Engineering and Physical Sciences Research Council (EPSRC) of UK under Grant EP/E060722/01

Mathematical optimization is now widely regarded as an indispensable modeling and solution tool for the design of wireless communications systems. While optimization has played a significant role in the revolutionary progress in wireless communication and networking technologies from 1G to 5G and onto the future 6G, the innovations in wireless technologies have also substantially transformed the nature of the underlying mathematical optimization problems upon which the system designs are based and have sparked significant innovations in the development of methodologies to understand, to analyze, and to solve those problems. In this paper, we provide a comprehensive survey of recent advances in mathematical optimization theory and algorithms for wireless communication system design. We begin by illustrating common features of mathematical optimization problems arising in wireless communication system design. We discuss various scenarios and use cases and their associated mathematical structures from an optimization perspective. We then provide an overview of recently developed optimization techniques in areas ranging from nonconvex optimization, global optimization, and integer programming, to distributed optimization and learning-based optimization. The key to successful solution of mathematical optimization problems is in carefully choosing or developing suitable algorithms (or neural network architectures) that can exploit the underlying problem structure. We conclude the paper by identifying several open research challenges and outlining future research directions.

Using diploid representation with dominance scheme is one of the approaches developed for genetic algorithms to address dynamic optimization problems. This paper proposes an adaptive dominance mechanism for diploid genetic algorithms in dynamic environments. In this scheme, the genotype to phenotype mapping in each gene locus is controlled by a dominance probability, which is learnt adaptively during the searching progress. The proposed dominance scheme isexperimentally compared to two other schemes for diploid genetic algorithms. Experimental results validate the efficiency of the dominance learning scheme

Logistics optimization nowadays is becoming one of the hottest areas in the AI community. In the past year, significant advancements in the domain were achieved by representing the problem in a form of graph. Another promising area of research was to apply reinforcement learning algorithms to the above task. In our work, we made advantage of using both approaches and apply reinforcement learning on a graph. To do that, we have analyzed the most recent results in both fields and selected SOTA algorithms both from graph neural networks and reinforcement learning. Then, we combined selected models on the problem of AMOD systems optimization for the transportation network of New York city. Our team compared three algorithms - GAT, Pro-CNN and PTDNet - to bring to the fore the important nodes on a graph representation. Finally, we achieved SOTA results on AMOD systems optimization problem employing PTDNet with GNN and training them in reinforcement fashion. Keywords: Graph Neural Network (GNN), Logistics optimization, Reinforcement Learning