(Tested on a Mac and connecting to Ubuntu server on AWS)

Assuming that you are using key-based authentication and have added your key to the remote server at ~/.ssh/authorized_keys and have the .pem  key file on your local machine - if none of this sound familiar first start here.

Next, you need to install the Remote Development extension pack if you do not already have it yet.

Now, click the 'remote window' bottom left of the window

This will open the command palette with the following options:

Select 'Remote-SSH: Open Configuration File' which will ask you to locate the configuration file on your local machine.

After you've selected the file enter the following details into it:

Host my_remote_server
    HostName 12.34.567.891 #the IP of your server
    User roy #the name of the user on the remote
    IdentityFile /Users/roygranit/ssh/key.pem #route of your key

And finally save this file. Now go back to command palette and now select the top option:

And select the name of the server you wish to connect to..

If all went smoothly you should now be connected the remote host