Comment by TheCoelacanth
3 years ago
Even if AGI is never achieved, it could still be an existential risk.
Something significantly stupider than an average human, but that was 100% focused and 100% loyal could potentially be used by a very smart human in a way that effectively made them super-intelligent to compared to an unaugmented human.
Computers have approximately 100% perfect memory recall, vastly increased factual and numeric memory compared to humans, much better calculation, 100% focus and "loyalty". I've been wondering whether a computer recognising a face as someone from your contacts counts as you being super-intelligent (I don't think it does), or a spreadsheet adding up thouands of numbers (also no), an infinite ToDo list and calendar reminders (maybe?), spaced repetition learning (possibly?) and from there - what would count? What would it mean for you to be super-intelligent by machine augmentation? What would computer software which effectively increased human intelligence look and behave like? Surely not like a window with text input and clicky buttons...?