Reddit mentions of Practical Optimization

Transistor-level IC designer.

The elective that benefited me the most was "Minimization of Functions" in the Applied Math department. This is a course in nonlinear optimization, where you learn how to find (numerically) the maxima and minima of highly nonlinear functions. It has been incredibly useful throughout my career, even in the simplest cases a/k/a curve-fitting.

The follow-on course, "Minimization of Functionals" was for the aero/astro Optimal Control types, people who want to shoot down ballistic missiles using ballistic missiles -- although the course catalog expressed it slightly differently. I skipped that one.

The book by Gill, Murray, and Wright gives a wonderful overview of the field (amazon link) but I recommend you buy a used copy or a previous-edition used copy. There ain't much that's changed in 40 years, except the cost of a trillion floating point operations has fallen by a factor of a million.

Since you are a practicing engieer with plenty of experience, I will suggest the right way to learn rather than the speed-of-the-internet , show-me-a-web-page way to acquire jargon.

Buy and read textbooks.

Start with Numerical Recipes by Press et al (Link 1). It has a couple of chapters on optimization and some very VERY excellent discussion. It will teach you the way academics formulate these problems, and how they solve them today.

Then read Gill, Murray, and Wright "Practical Optimization" (Link 2).

Next comes Roger Fletcher, "Practical Methods of Optimization". This book has been published two different ways: as a single volume, and also split into two volumes. Since Amazon Used Books sells the two volumes for considerably less money, I recommend that path: (Link 3) and (Link 4) .

After you have read those books, you will be able to appreciate the following paragraph:

I myself have found, in practice, that some of the old 1960's approaches to optimization work DELIGHTFULLY WELL on 2015 real world engineering problems, using 2015 computer power. In fifty years the problems have become 10,000 times more difficult and the computers have become 2^(50/3) times more powerful. The computers are winning the tug of war.

Make an honest try to solve your problem using no-derivative unconstrained optimizers, plus penalty functions or barrier functions for the constraints. I think you will be very pleasantly surprised. If you have honestly done your best and tried your hardest to get this to work, and failed, then your fallback is to implement the full stochastic miasma. Start with the TOMS paper by Corana, Marchesi, Martini, and Ridella. It is the most engineering-results oriented discussion I know of. If you are a masochist, try (just try!) to read the various publications and white papers by Lester Ingber. You will regret it.