This book is a solution manual of the 110 questions that were presented in the eight chapters throughout the author’s previous book, Optimal Control Engineering with MATLAB. The manual is prepared as an independent book since it includes the problems’ questions, figures, and all necessary equations and formulas in a summarizing fashion and within the given solution frameworks. The idea behind writing this book is to increase significantly the student’s or reader’s knowledge about the optimal control discipline that is found in many engineering fields. Detailed step-by-step hand solutions are carried out while utilizing the MATLAB functions in the toolboxes. Simulink models are presented to simulate and validate the obtained results. Each chapter concludes with the lists of MATLAB programs that are used either to assess or to develop the desired solutions to challenging problems. The reader will discover the importance of using these programs, especially in solving parameter optimization design, different nonlinear TBVP, finite horizon LRQ, DDP and ILRQ problems. In addition, the implementation of the obtained time function optimal control law by a transfer function controller is accomplished for some problems. Among these problems, the reader will find many practical engineering problems in continuous as well as discrete forms. Finally, the reader can easily modify these programs and Simulink models for his or her problems.