svn+ssh and remembering location of private key

I there a way to have IDEA 5.x remember the location of my SSH private key without the passphrase when using the SVN integration? Similarly to how it works with CVS.

In order words, I don't want to specify the location of my private key every time, but I also don't want it to "remember" my passphrase.

I seems to be all or nothing.

Thanks,

E.