Comment by maciejzj

I would agree that a decent EE major is equipped with knowledge sufficient to grasp both Transformer architecture and gradient descent training. However, these elements do not fully cover the key parts of modern AI systems and also are not enough to predict the AI capabilities. You have to account for reinforcement learning from human feedback (RLHF) and with verifiable rewards (RLVR) which are crucial elements of the wider model creation/training process. These do not have obvious analogues in other engineering disciplines, but are still within your grasp if you are decent CS/EE/adjacent major. The issue is that you still can't deduce end result AI capabilities from first principles. You may have prefect understand of y = softmax(QK^T/sqrt(d_k))*V and it will give you no clue of the effectiveness of a given AI model. The range and complexity of training RLVR tasks may hint at the model capabilities, but the final results are very much emergent.