How to Connect two Linux machines using SSH keys

If you are given a task to check the CPU and memory utilization then login into 5 different machines becomes a tedious task and you are overwhelmed. SSH Key-based authentication is one of the safest and best to solve the connectivity problem.

Stay tuned to this tutorial and this guide will help you forever with SSH connectivity. So let’s start !!

What is SSH Protocol?

SSH is a secure shell protocol to connect Linux machines securely and remotely. When you connect to a remote machine, you log in using an account that already exists. When you connect to the remote machine you are dropped into a shell session which is a text-based interface and also to achieve this your machine must have an ssh client installed and SSH Daemon must be running on the remote machine.

There are two ways in which you connect to a remote machine one is by using a username/password which is not a safer method and the other method is to use SSH Keys which is secure.

What are SSH KEYS ?

SSH keys are sets of matching cryptographic keys which can be used for authentication. One of them is the Private key which is never shared with anyone and kept for your login purpose and the other is Public Key which can be shared. Let’s get started and learn how to generate SSH Keys

Generating new SSH Key Pair and Connecting to remote machine using SSH keys

Generating the new SSH key Pair is straightforward and to generate SSH key’s there are multiple cryptographic algorithms such as RSA , DA and ECDSA that can be used and one of the majorily and preferred is RSA.

  • To generate the SSH keys run the ssh-key command as shown below. After you execute the command two keys are generated one is private and the other is public key.
  • When prompted select all the values as default and continue hitting enter button.
  • Both the keys (Private and public keys) will be generated under ~/.ssh with the following names id_rsa which is Private key and is Public Key
  • Next, copy the Public Key to another remote machine by using ssh-copy-id command.
  ssh-copy-id ubuntu@remotemachine

If you get Permission denied (publickey) while running ssh-copy-id ubuntu@remotemachine then on remote node edit the /etc/ssh/sshd_config file and update PasswordAuthentication from no to yes then restart the service using sudo systemctl restart sshd command.

  • After the ssh-copy-id command is succesfully executed you will see the public key is copied under ~/.ssh/authorized_keys directory in remote machine.

Note: If you wish to copy the public key without ssh-copy-id command then use cat ~/.ssh/ | ssh ubuntu@remotemachine “mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys”

  • Now try logging into remote machine without password using the below command.
ssh ubuntu@remotemachine

As you can see below you are able to connect to remote machine using SSH Keys.

Generating Key Pair in AWS account and Connecting two remote AWS EC2 instances with same key Pairs

Earlier in previous section you learned how to generate keys in Linux machine and then connect two linux machines. Now, in this section learn how to generate key pair in AWS account and Connecting two remote AWS EC2 instances with same key Pairs. To do that follow the below steps:

  • First thing is to create a new KeyPair under EC2 dashboard with any name of your choice such as “newly”
  • Download the key in pem format which you just created that is newly.pem.
  • Next, Launch two EC2 instance from AWS console named source instance and remote instance with same keypair (“newly”)
    • IP address of Source in my case is :
    • IP address of destination in my case is :
  • Now, convert newly.pem into newly.ppk using puttygen tool which you will use to login to source machine and remote machine.
  • Login to source machine using Public IP address with newly.ppk key.
  • Further under your home directory create a file named awskeypair .
  • Paste the content of newly.pem key from your local machine to awskeypair file as shown below.
  • Modify the permissions of awskeypair file to 600 with below command.
chmod awskeypair 600
  • Now you are at final stage and lets run the below ssh command
ssh -i awskeypair ubuntu@ 

As you can see below you are able to connect to remote machine using SSH Keys again.

Important SSH commands

  • Connect to Remote server
    • ssh remotemachine
  • Connect to Remote server with different user on remote machine
    • ssh user@remotemachine
  • Connect to Remote server and run command
    • ssh user@remotemachine command-to-run
  • Connect to Remote server if it runs on NON standard Port 22
    • ssh -p port_number user@remotemachine
  • Disable Password Authentication
    • Modify /etc/ssh/sshd_config
    • and Mark PasswordAuthentication as NO
    • Run command service ssh restart
  • Changing the SSH Daemon Port on which it runs
    • vi /etc/ssh/sshd_config
    • Comment out Port 22 and add desired Port
    • Run command service ssh restart
  • Limit Users or Groups to Login
    • vi /etc/ssh/sshd_config
    • Search for Allowusers and then append usernames in front of it ( For users )
    • Search for AllowGroups and then append group-names in front of it ( For Groups )
    • Run command service ssh restart
  • Disable Root Login
    • vi /etc/ssh/sshd_config
    • PermitRootLogin no
    • Run command service ssh restart


In this tutorial you learnt how to generate and use SSH keys to securely login to remote machines. With SSH keys you can secure your connections and doesn’t require you to enter password every time you login to your remote machines. This is very essential for running Ansible Playbooks as well.

One thought on “How to Connect two Linux machines using SSH keys

  1. Pingback: How to Install Ansible and Run ansible-playbooks on Ubuntu 18.04 LTS

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s