Add ssh-agent support

[pmeinhardt]

It would be nice to add support for ssh-agent, so SSHKit can fetch keys stored by it.

Update: Since our changes for adding agent-support to OTP have made it into OTP 23, we could add an example script or at least document how to use it with SSHKit connections. ✌️

http://erlang.org/doc/man/ssh_agent.html

[Markcial]

I started a little project, i don't know if it might help, https://github.com/Markcial/spy

[pmeinhardt]

Cool, thanks a lot @Markcial. I'll definitely need take a more detailed look. Maybe @tessi or @brienw have some thoughts on this topic as well? 😁

The initial question for me would be whether we'd want agent support to rely on ports and shell commands.

[brienw]

tagging @holetse and @rjanja as well, we've been discussing this more and more recently and would love to get some sort of ssh-agent support added to either SSHKit or the ssh_client_key_api, We'll take a look at @Markcial's project. I think ultimately it would be nice to not have to rely on shell commands, but when we've previously attempted to jump down that rabbit hole ourselves it's been a very dark scary place.

[pmeinhardt]

@brienw I couldn't sleep, so I decided to go to that "very dark scary place" and found some light:
https://gist.github.com/pmeinhardt/8c746ebe8d7397a2a07dabb1ba7ab30c 💡

With this, I was able to connect to the OpenSSH ssh-agent running on my machine and retrieve the added public keys. They should be printed base64 encoded along with the comment (which is the key file path here).

This could be a starting point 😁

[Markcial]

I am not very proficient on erlang, but cannot be possible to use this library for plain key management? http://erlang.org/doc/man/public_key.html#der_decode-2

[pmeinhardt]

Awesome @Markcial, I think that's used in labzero/ssh_client_key_api as well. So maybe we can take a peek there.

I need to refresh my memory on Erlang's ssh_client_key_api, but maybe there's a way of implementing that behaviour and internally communicating with the ssh-agent process as outlined in the gist to add agent support to SSHKit.

I'm hooked 🎣

[brienw]

Nice @pmeinhardt! If i recall, one of the scary places was in matching the loaded keys to the one needed for the server being connected to. If there are numerous keys, there didn't appear to be an obvious way to match the keys, and just brute-force "try them all until one works" didn't seem like a good strategy to us. It's been a while since any of us looked at this, so i don't remember exactly what the pain point was. Probably worth looking into again now

[pmeinhardt]

Mmh, I see @brienw. Need to do a bit more reading, but here are some early findings 🔎

1. Going through all keys does not appear to be an absurd idea:
   • paramiko/client.py:642-658
   • demos/demo.py:40-58
2. Here's how the list of identities are retrieved from the agent, which is the same procedure that's performed in the gist I posted above: paramiko/agent.py:64-73
3. Internally though, the authentication is then coordinated by a series of SSH protocol messages:
   • …kicked off here: paramiko/auth_handler.py:96-105
   • then, the client issues an SSH service request "ssh-userauth" to the server
   • if that's accepted, the client sends a SSH_MSG_USERAUTH_REQUEST with
     • the username,
     • service name ("ssh-connection"),
     • method name ("publickey") and
     • a number of publickey auth-specific fields, namely:
       • true,
       • public key algorithm name,
       • public key to be used for authentication and
       • signature which let's the agent sign a blob (includes session id, username, …)
   • here's how paramiko constructs the MSG_USERAUTH_REQUEST: paramiko/auth_handler.py:261-274

I'm not yet sure whether/how we can fit that into the Erlang APIs and need to head out… I'll take another look later. Any insights/ideas are welcome of course 👋

Note to self: A diagram may be helpful…
Note to self: RFC 4252

[pmeinhardt]

Hi there everyone, sorry for the cliffhanger! So here's how I currently understand…

The problem

While it is possible to:

1. talk to the ssh-agent and retrieve a list of public keys using a SSH2_AGENTC_REQUEST_IDENTITIES request and decoding the response (see gist)
2. talk to the ssh-agent and have it sign data using a specific key by sending an SSH2_AGENTC_SIGN_REQUEST

…it seems like we're missing a hook in the Erlang APIs that would allow us to hook into the authentication the way that we would need to.

The Erlang key_cb option for ssh:connect/2,3 expects us to return a private key. However, one of the main principles of ssh-agent is to never expose private keys. This is where we're stuck until Erlang offers more powerful callback options to let the agent do the signing instead.

Take a look at ssh_auth:publickey_msg/1 & ssh_auth:get_public_key/2 to see how key_cb/ssh_client_key_api come into play when authenticating using "publickey" authentication. I don't see a way of getting ssh-agent support using the current Erlang SSH modules.

Please let me know if I misinterpreted any of the available information or you see a different way of using an ssh-agent auth flow with the existing APIs.

Options

So, how do we move forward? Here's a few options I can think of…

1. read private SSH keys from ssh-agent process memory… just kidding 😉
2. implement our own SSH stack (e.g. extracting the ssh code from Erlang, adapt it to our needs and build on top of that instead of the built-in version) 😒
3. try to contribute to Erlang/OTP to add options to hook into the SSH authentication flow and have a different process, the ssh-agent, sign the data (which means only future Erlang/OTP releases would support this) 🤔
4. spawn a native ssh binary which supports ssh-agent and connect to that process from Elixir (which comes with its own set of problems of course) 😱
5. not support ssh-agent 😞
6. other options…??

Let me know if you see any other options. It's already late again, so it's likely I missed alternatives to explore…

At the moment, I feel like creating a PR on the erlang/otp repo to see where it leads us. I'll check the contribution guidelines and whip up a PR to get feedback… not today though.

[pmeinhardt]

A brief update: I am in touch with the great developers from the Erlang team ❤️ and I am hoping to maybe get SSH Agent support into Erlang/OTP. It'll be a bit though before I have more tangible results ✌️

[pmeinhardt]

📻 Status update: I have created a work-in-progress PR on the OTP project last week.
It has already received some feedback and it looks like it might make it in. 🤞

Note that this is still a draft and the API might change.

In its current state, there will be a new built-in ssh_client_key_api module:

context = SSHKit.context("example.com", key_cb: {:ssh_file_with_agent, []})
# or:
conn = SSHKit.SSH.connect("example.com", key_cb: {:ssh_file_with_agent, []})

I think this'd be a great default for Bootleg @brienw, @holetse, @rjanja ✌️

[pmeinhardt]

Here's a link to the PR: erlang/otp#2298

[pmeinhardt]

FYI: @brienw, @holetse, @rjanja, we managed to get SSH agent support into OTP. 🎉

Not 100% sure yet, when it will be officially released.
I think we were slightly too late to make it into the 23.0 release.

Here's a link to the documentation:
https://github.com/erlang/otp/blob/master/lib/ssh/doc/src/ssh_agent.xml

So, in short, you can use it like this:

context = SSHKit.context("example.com", key_cb: {:ssh_agent, []})
context = SSHKit.context("example.com", key_cb: {:ssh_agent, [socket_path: SocketPath]})

— erlang/otp#2554

[pmeinhardt]

Note to self: We should probably provide an example of how to use the new behavior in the SSHKit documentation once it lands in the next OTP release. 📝