I do all my web development ssh-ing to a Linux box. On my server the Mac ‘delete’ button worked fine in shell, but was deleting forward in Vim.
Here’s a fix!
Run a command in your terminal window:
stty -a | grep erase
and check if it says
erase = ^?;
If it is set to something else (e.g.
^H) then put the following line in both
.bashrc and in either
if tty --quiet ; then stty erase '^?' fi
Taken from http://www.ibb.net/~anne/keyboard.html