Enter SSH passphrase only once

$ ssh-add /home/username/.ssh/id_rsa

Use absolute path instead of relative path to avoid re-adding key after reboot.

SSH VPS without password

ssh-keygen -t rsa

scp id_rsa.pub username@server:~/.ssh/id_rsa.pub
> ssh username@server
> cd ~/.ssh
> cat id_rsa.pub >> authorized_keys
> exit

ssh username@server

Reference

Multiple SSH Keys for Multiple Github Accounts
askubuntu
perminent add ssh key on stackoverflow