The journey began with an exhaustive analysis of current optimizers, identifying their strengths and weaknesses. They noticed that while Adam was excellent for many tasks due to its adaptive learning rate for each parameter, it sometimes struggled with convergence on certain complex problems. On the other hand, SGD, while simple and effective, often required careful tuning of its learning rate and could get stuck in local minima.