You can create a new key pair and delete the old ones so the new servers will start using the new key (you will need to select it). The problem is that the already running servers probably have this key in the authorized_keys file so that guy can access those servers with the current key.
The easiest way to solve this issue is to create your own key pair and then import that one using the AWS console
You will need to select that key when launching new servers but for the running ones, you will need to access them using a SSH connection and the public key to the .ssh/authorized_keys file
Please take into account that if you do something wrong when modifying that file, you won't be able to access the server anymore so ask us any questions you have before modifying any configuration in the server.