IPSec secured L2TPv3 - one way traffic in L2 tunnel

Question

Sigh... after 7 hours battling coming here because I've exhausted all my options to find an answer for my problem.
So here is the topology - standard (boring) IPSec secured L2TPv3 tunnel: on one side - 897 connected to a DSL box, on another side - 1921 with two interfaces.
Purpose to setup a plain L2TPv3 tunnel between those locations so computers plugged into the 897's 8-port switch interface can communicate with number of devices connected to 1921 on other side. 
897:
crypto ikev2 keyring key1
 peer destination_ip_address
  address local_outside_ip_address
  pre-shared-key key
crypto ikev2 profile default
 match identity remote address 1921_outside_ip_address 255.255.255.255
 identity local address 897_outside_ip_address
 authentication remote pre-share
 authentication local pre-share
 keyring local key1
crypto ikev2 dpd 30 3 periodic
controller VDSL 0
ip ssh rsa keypair-name router-key
ip ssh version 2
pseudowire-class DZD
 encapsulation l2tpv3
 ip local interface Loopback1
 ip pmtu
 ip dfbit set
 ip tos reflect
crypto ipsec transform-set default esp-aes esp-sha-hmac
 mode tunnel
crypto ipsec df-bit set
crypto map local 1 ipsec-isakmp
 set peer 1921_outside_ip_address
 set ikev2-profile default
 match address 130
interface Loopback1
 ip address 172.16.1.1 255.255.255.255
interface ATM0
 no ip address
 no atm ilmi-keepalive
interface Ethernet0
 no ip address
interface GigabitEthernet0
 no ip address
interface GigabitEthernet1
 no ip address
interface GigabitEthernet2
 no ip address
interface GigabitEthernet3
 no ip address
 xconnect 172.16.1.2 1 encapsulation l2tpv3 pw-class DZD
interface GigabitEthernet4
 no ip address
interface GigabitEthernet5
 no ip address
interface GigabitEthernet6
 no ip address
interface GigabitEthernet7
 no ip address
interface GigabitEthernet8
 no ip address
 duplex auto
 speed auto
 pppoe enable group global
 pppoe-client dial-pool-number 1
interface Wlan-GigabitEthernet8
 no ip address
interface wlan-ap0
 description Embedded Service module interface to manage the embedded AP
 ip unnumbered Vlan1
interface Vlan1
 ip address 10.97.2.29 255.255.255.0
interface Dialer1
 mtu 1492
 ip address negotiated
 ip nat outside
 ip virtual-reassembly in
 encapsulation ppp
 ip tcp adjust-mss 1452
 dialer pool 1
 dialer-group 1
 ipv6 address autoconfig
 ppp authentication pap callin
 ppp pap sent-username DSL_username password DSL_password
 crypto map local
ip forward-protocol nd
ip http server
no ip http secure-server
ip route 0.0.0.0 0.0.0.0 Dialer1
access-list 130 permit ip host 172.16.1.1 host 172.16.1.2
dialer-list 1 protocol ip permit
c897#
1921:
crypto ikev2 keyring key1
 peer 897_outside_ip_address
  address 897_outside_ip_address
  pre-shared-key key
crypto ikev2 profile default
 match identity remote address 897_outside_ip_address 255.255.255.255
 identity local address 1921_outside_ip_address
 authentication remote pre-share
 authentication local pre-share
 keyring local key1
crypto ikev2 dpd 30 3 periodic
ip ssh version 2
lldp run
pseudowire-class ZRH
 encapsulation l2tpv3
 ip local interface Loopback1
 ip pmtu
 ip dfbit set
 ip tos reflect
crypto ipsec transform-set default esp-aes esp-sha-hmac
 mode tunnel
crypto ipsec df-bit set
crypto map local 1 ipsec-isakmp
 set peer 897_outside_ip_address
 set ikev2-profile default
 match address 130
interface Loopback1
 ip address 172.16.1.2 255.255.255.255
interface Embedded-Service-Engine0/0
 no ip address
interface GigabitEthernet0/0
 description WAN-ACC
 ip address 1921_outside_ip_address 255.255.255.0
 duplex auto
 speed auto
 crypto map local
interface GigabitEthernet0/1
 description LAN-Trunk
 no ip address
 duplex auto
 speed auto
 xconnect 172.16.1.1 1 encapsulation l2tpv3 pw-class ZRH
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 default_gateway_of_1921
logging host 10.96.2.21
access-list 130 permit ip host 172.16.1.2 host 172.16.1.1
pnc01921#
Note - 1921 is connected to the Nexus 2248TP FEX, here is the config of the interface of the FEX:
pnc00001# sh run int e101/1/6
!Time: Thu May  1 06:15:02 2014
version 5.0(3)N2(2b)
interface Ethernet101/1/6
  switchport access vlan 702
Now, IPsec tunnel comes up and does pass traffic - I can ping from one l1 another l1, below is the output from 897:
sh cry ike sa
 IPv4 Crypto IKEv2  SA
Tunnel-id Local                 Remote                fvrf/ivrf            Status
1         897_outside_ip_address/500     1921_outside_ip_address/500     none/none            READY
      Encr: AES-CBC, keysize: 256, Hash: SHA512, DH Grp:5, Auth sign: PSK, Auth verify: PSK
      Life/Active Time: 86400/76 sec
 IPv6 Crypto IKEv2  SA
#sh cry ips sa
interface: Dialer1
    Crypto map tag: local, local addr 897_outside_ip_address
   protected vrf: (none)
   local  ident (addr/mask/prot/port): (172.16.1.1/255.255.255.255/0/0)
   remote ident (addr/mask/prot/port): (172.16.1.2/255.255.255.255/0/0)
   current_peer 1921_outside_ip_address port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 7, #pkts encrypt: 7, #pkts digest: 7
    #pkts decaps: 51, #pkts decrypt: 51, #pkts verify: 51
    #pkts compressed: 0, #pkts decompressed: 0
    #pkts not compressed: 0, #pkts compr. failed: 0
    #pkts not decompressed: 0, #pkts decompress failed: 0
    #send errors 0, #recv errors 0
     local crypto endpt.: 897_outside_ip_address, remote crypto endpt.: 1921_outside_ip_address
     path mtu 1492, ip mtu 1492, ip mtu idb Dialer1
     current outbound spi: 0x852BF1F2(2234249714)
     PFS (Y/N): N, DH group: none
     inbound esp sas:
      spi: 0x5D9DFB1A(1570634522)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 2, flow_id: Onboard VPN:2, sibling_flags 80000040, crypto map: local
        sa timing: remaining key lifetime (k/sec): (4190855/3504)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)
     inbound ah sas:
     inbound pcp sas:
     outbound esp sas:
      spi: 0x852BF1F2(2234249714)
        transform: esp-aes esp-sha-hmac ,
        in use settings ={Tunnel, }
        conn id: 1, flow_id: Onboard VPN:1, sibling_flags 80000040, crypto map: local
        sa timing: remaining key lifetime (k/sec): (4190863/3504)
        IV size: 16 bytes
        replay detection support: Y
        Status: ACTIVE(ACTIVE)
#ping 172.16.1.2 sour l1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 172.16.1.2, timeout is 2 seconds:
Packet sent with a source address of 172.16.1.1
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/23/24 ms
Now, L2 tunnel shows to be up on both ends as well (output from 897 here)
#sh xconnect all
Legend:    XC ST=Xconnect State  S1=Segment1 State  S2=Segment2 State
  UP=Up       DN=Down            AD=Admin Down      IA=Inactive
  SB=Standby  RV=Recovering      NH=No Hardware
XC ST  Segment 1                         S1 Segment 2                         S2
------+---------------------------------+--+---------------------------------+--
UP     ac   Gi3(Ethernet)                UP l2tp 172.16.1.2:1                 UP
However, if you look at detailed output of l2tunn, you will see that the tunnel receives traffic from 1921, but does not send anything:
#sh l2tun tunnel all
L2TP Tunnel Information Total tunnels 1 sessions 1
Tunnel id 3504576447 is up, remote id is 2898810219, 1 active sessions
  Locally initiated tunnel
  Tunnel state is established, time since change 00:19:34
  Tunnel transport is IP  (115)
  Remote tunnel name is pnc01921
    Internet Address 172.16.1.2, port 0
  Local tunnel name is pnc0DRZD
    Internet Address 172.16.1.1, port 0
  L2TP class for tunnel is l2tp_default_class
  Counters, taking last clear into account:
    0 packets sent, 763 received
    0 bytes sent, 65693 received
    Last clearing of counters never
  Counters, ignoring last clear:
    0 packets sent, 763 received
    0 bytes sent, 65693 received
  Control Ns 18, Nr 9
  Local RWS 512 (default), Remote RWS 512 (max)
  Control channel Congestion Control is disabled
  Tunnel PMTU checking enabled
  Retransmission time 1, max 1 seconds
  Unsent queuesize 0, max 0
  Resend queuesize 0, max 2
  Total resends 0, ZLB ACKs sent 8
  Total out-of-order dropped pkts 0
  Total out-of-order reorder pkts 0
  Total peer authentication failures 0
  Current no session pak queue check 0 of 5
  Retransmit time distribution: 0 0 0 0 0 0 0 0 0
  Control message authentication is disabled
Mirrored situation on other side - 1921 sends packets, but nothing is received:
pnc01921#sh l2tun tunnel all
L2TP Tunnel Information Total tunnels 1 sessions 1
Tunnel id 2898810219 is up, remote id is 3504576447, 1 active sessions
  Remotely initiated tunnel
  Tunnel state is established, time since change 00:21:15
  Tunnel transport is IP  (115)
  Remote tunnel name is pnc0DRZD
    Internet Address 172.16.1.1, port 0
  Local tunnel name is pnc01921
    Internet Address 172.16.1.2, port 0
  L2TP class for tunnel is l2tp_default_class
  Counters, taking last clear into account:
    815 packets sent, 0 received
    69988 bytes sent, 0 received
    Last clearing of counters never
  Counters, ignoring last clear:
    815 packets sent, 0 received
    69988 bytes sent, 0 received
  Control Ns 9, Nr 20
  Local RWS 1024 (default), Remote RWS 512
  Control channel Congestion Control is disabled
  Tunnel PMTU checking enabled
  Retransmission time 1, max 1 seconds
  Unsent queuesize 0, max 0
  Resend queuesize 0, max 1
  Total resends 0, ZLB ACKs sent 18
  Total out-of-order dropped pkts 0
  Total out-of-order reorder pkts 0
  Total peer authentication failures 0
  Current no session pak queue check 0 of 5
  Retransmit time distribution: 0 0 0 0 0 0 0 0 0
  Control message authentication is disabled
There is a Windows box plugged into 897's G3 with IP address 10.97.2.25. I can ping from it 897's VLAN1 at 10.97.2.29. However I can't ping anything across the L2TPv3 tunnel. At the same time on that Windows box I can see broadcast traffic coming across the tunnel.
I give up. Anyone has some reasonable suggestion what might be wrong? I suspect that something is wrong at 897's side. 
One last question - how can I create svi on 1921 and assign ip address from 10.97.2.0/24 network on it?

Answer 1

Anybody? Opened ticket #630128425, no response from Cisco yet..