IpSec in transport mode not completeing phase 2 quick mode

Question

I am trying to connect Solaris 10 to a non Solaris box over IPsec. I know this box has worked with a windows machine running ipsec.
My configuration of ipsec and ike looks to be correct but I must be missing something.
If I turn the ikeadm traces I see that I get through the phase 1 main mode but can not establish quick mode.
I have to use 3des for encryption and sha1 for authentic ion. I have followed the steps in the Solaris 10 ipsec and ike manual but I don't know where to turn now.
This are the ikeadm traces
Wed 20 Jul 2005 12:05:21 BST: in.iked: Quick Mode negotiation failed: code 14 (No proposal chosen).
Wed 20 Jul 2005 12:05:21 BST: in.iked: local_ip = 172.18.10.1, remote_ip = 192.168.25.22,
Wed 20 Jul 2005 12:05:21 BST: in.iked: local_i_id = ipv4(tcp:0,[0..3]=192.168.25.22), local_r_id = No Id,
Wed 20 Jul 2005 12:05:21 BST: in.iked: remote_i_id = ipv4(tcp:2126,[0..3]=172.18.10.1), remote_r_id = No Id,
Wed 20 Jul 2005 12:05:21 BST: in.iked: spsi: ike_send_packet -1
Wed 20 Jul 2005 12:05:21 BST: in.iked: In ssh_policy_phase_ii_sa_freed.
Wed 20 Jul 2005 12:05:21 BST: in.iked: local_ip = 172.18.10.1, remote_ip = 192.168.25.22,
Wed 20 Jul 2005 12:05:21 BST: in.iked: spsi: ike_udp_callback_common -1
Wed 20 Jul 2005 12:05:21 BST: in.iked: In ssh_policy_new_connection_phase_qm (pm_info = 0x85938).
Wed 20 Jul 2005 12:05:21 BST: in.iked: In ssh_policy_qm_select_sa (pm_info = 0x85938).
Wed 20 Jul 2005 12:05:21 BST: in.iked: Number of sas is 1.
Wed 20 Jul 2005 12:05:21 BST: in.iked: pfkey_request: queueing seq 598 type 12/X_INVERSE_ACQUIRE satype 0/UNSPEC
Wed 20 Jul 2005 12:05:21 BST: in.iked: tx_req: posting seq 598 type 12/X_INVERSE_ACQUIRE satype 0/UNSPEC
Wed 20 Jul 2005 12:05:21 BST: in.iked: pf_key_handler: got pid 1242 seq 598 type
6/ACQUIRE sa 0/UNSPEC errno 0 diag 0/No diagnostic len 109
Wed 20 Jul 2005 12:05:21 BST: in.iked: handle_reply: got seq 598 type 6/ACQUIREsatype 0/UNSPEC
Wed 20 Jul 2005 12:05:21 BST: in.iked: SA #0.
Wed 20 Jul 2005 12:05:21 BST: in.iked: Number of proposals = 1.
Wed 20 Jul 2005 12:05:21 BST: in.iked: Proposal 0.
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 0 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 1 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 2 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 3 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 4 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 5 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 6 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 7 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 8 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: ecomb 9 lost
Wed 20 Jul 2005 12:05:21 BST: in.iked: no matching ecomb
Wed 20 Jul 2005 12:05:21 BST: in.iked: No winner.
Wed 20 Jul 2005 12:05:21 BST: in.iked: finish_qm_select_sa: invoked for 85018
Wed 20 Jul 2005 12:05:21 BST: in.iked: Quick Mode negotiation failed: code 14 (No proposal chosen).
Wed 20 Jul 2005 12:05:21 BST: in.iked: local_ip = 172.18.10.1, remote_ip = 192.1
My config file
# more config
local_id_type IP
p1_lifetime_secs 28800
p1_nonce_len 20
p1_xform {auth_method preshared oakley_group 2 auth_alg sha1 encr_alg 3des}
p2_pfs 0
p2_lifetime_secs 10800
label "cmts1"
local_addr 172.18.10.1
remote_addr 192.168.25.22
I have also set up the ike.preshared file with my preshared key
When I do a ikeadm dump preshared I see the correct key
Any suggestions
penright

Answer 1

Eeesh. I wish I'd signed up for SDN earlier.
I know I'm a year late in replying, but the peer is proposing something in Quick Mode
(Phase 2) that your Solaris box doesn't think is available. Given the combinations
you had (0-9), I'd be interested to know what the peer proposed that didn't match.
You don't mention what ipsecconf(1m) input is, nor what the peer is configured to
do. You say 3des + sha1 - so that should be one of the choices.
One common mistake is to use "auth_algs" in ipsecconf(1m) (which is AH) instead
of "encr_auth_algs" (which is ESP's hash).