WARNING -> Please be careful when reading this problem description. I had some assumptions that were not correct as I was writing this question. Make sure you read my answer explaining what I had wrong!

I have host A in AWS as an EC2 instance.

I a private key embedded in a .PEM file that I can use to host A with SSH key Z. This works if I pass it to the ssh command using the -l argument, AND if I turn off strict host checking with -o StrictHostKeyChecking=no.

I would strongly prefer to leave strict host checking on even though I "know" this is the correct host because I'm interacting with the AWS interface, getting the ip/dns from them, and I'm inside of my own little VPC world.

It seems like the only way to provide the fingerprint -> host mapping is by providing it in a known_hosts file.

Is that correct?

If that is correct, how can I take the private key embedded in the .PEM file that I have from AWS and build the correct entry for the single fingerprint -> host mapping for a temporary known_hosts file that I can read when I'm logging into the EC2 instance?

WHAT I DO NOT WANT TO DO

• Use ssh-keyscan. All this does is blindly accept the fingerprint of the remote client without validating that it matches with the key. I think?


• Turn off StrictHostKeyChecking. I want to establish good practices early, and I need to know how to do this now, because I'm going to need to know how to do this in general. (By this I mean how to use SSH fingerprints to validate the identity of the host I'm connecting to, based on the key that I have.)


• Mess around with ssh-add. I want to write this to a file that's easy to lockdown access to, not put it into a running process.

EDITS:
Strangely when I try to extract the fingerprint from the pem file it doesn't match the fingerprint I see when I connect and it prompts me.

FINGERPRINT EXTRACTION FROM PEM

bash-4.2$ ssh-keygen -l -E md5 -f ./blah.PEM
2048 MD5:be:b1:d7:e1:f0:0f:ce:41:60:fa:97:dc:b8:2c:ed:08 no comment (RSA)
bash-4.2$ ssh-keygen -l -E sha1 -f ./blah.PEM
2048 SHA1:g2PDmIcw19Z/v7HTco6xRWxQ88c no comment (RSA)

FINGERPRINT DISPLAY DURING SSH PROMPT

bash-4.2$ ssh -i ./blah.PEM ubuntu@ip-172-31-6-91.us-east-2.compute.internal
The authenticity of host 'ip-172-31-6-91.us-east-2.compute.internal (172.31.6.91)' can't be established.
ECDSA key fingerprint is SHA256:ibwhkrF5oMapJla4cKuXgePT5lHmg08L7yMp6auCpgo.
ECDSA key fingerprint is MD5:ba:82:53:ee:89:22:26:63:26:11:21:93:63:1f:1d:d1.

How could the fingerprints be different, but the key still allows me to connect?

You have 2 key pairs at play there:

1. Server's Private/Public key.

ssh daemon on the server has a set of private keys created and stored in the /etc/ssh/ folder

The RSA fingerprint you are getting from the server comes from the public key corresponding to the /etc/ssh/ssh_host_rsa_key private key

2. User's Private/Public key.

This is a keypair you own. The private key should be securely stored on your computer and used to authenticate to the server. The public key's fingerprint is on the server, in your profile's authorized_keys file: ~/.ssh/authorized_keys

So there are 2 different public keys, and their fingerprints will not match, unless you use the same private key as one on the server, which is unlikely.

To get rid of warning do exactly as it has been asking: put fingerprint of the server into the /var/lib/jenkins/.ssh/known_hosts file.

If I understand you correctly, the private key file is in your possession and you'd like to get the fingerprint of it so that you can add it into your known_hosts file. If that's right, then here's what you do:

$ ssh-keygen -yf /path_to_private_key/key_file_name

That will output something like:

ssh-rsa AAAAB3NzaC....

Lastly, prefix that with the IP address to which you SSH, so that you have this:

10.200.25.5 ssh-rsa AAAAB3NzaC....

and you can add that as a line in your known_hosts file.

My underlying confusion was that I thought I had the exact same pair of private and public keys that the server did. Instead what's happening is when I create a key pair and assign it to a new EC2 instance, the EC2 instance is getting the public key of that pair put into its authorized_keys which allows me to connect to it with the private key that I download when creating the pair in AWS.

I can use the fingerprinting command that comes with AWS, but it's only good to validate that the private key that I have, matches the public key they have stored, and will put into the authorized_keys.

Every time a new EC2 instance comes up, it generates a collection of its own private/public keys for different algorithms like RSA and DSA. I must now scrape the logs to get the fingerprints for those keys so that I can validate that they match the host I'm connecting to.

So the steps are.

1. Generate the EC2 instance, keep the key you get.


2. Give that key from step 1 to Jenkins so that it can connect to the host.


3. Use the get-console-output command to scrape the fingerprints for the keys from the logs.


4. Attempt to connect to the remote instant with the key from step 1. Use the key fingerprint from that error message to validate against the fingerprint you scraped in step 3.


5. Once you've validated, then you know it's safe to add the remote host.


6. Profit!!!

Keep in mind the vital issue here is that you can't trust that the host you're connecting to isn't a man in the middle attack. If you blindly accept the key without validating it's fingerprint in step 4, you may not be connecting to the server you expect to be. By validating in step 4 you know that your connection is secure (because of SSH's cryptography), but crucially you also know WHO you are connected to, because only one person is going to have the key-pair fingerprint matching the one you expect.