Caveats
The primary caveat of overvolting is increased heat: the power dissipated by a circuit increases with the square of the voltage applied, so even small voltage increases significantly affect power. At higher temperatures, transistor performance is adversely affected, and at some threshold, the performance reduction due to the heat exceeds the potential gains from the higher voltages. Overheating and damage to circuits can occur very quickly when using high voltages.
There are also longer-term concerns: various adverse device-level effects such as hot carrier injection and electromigration occur more rapidly at higher voltages, decreasing the lifespan of overvolted components.
Read more about this topic: Dynamic Voltage Scaling