Exploring use of Windows 10 VPN client to access Cisco AnyConnect IPsec/IKEv2

The Cisco AnyConnect client supports two VPN transports: SSL (TLS plus optionally DTLS) and IPsec/IKEv2. The IPsec/IKEv2 connection transport is standard and AnyConnect seemingly just differs from the Windows VPN client in so far as it supports a Cisco specific EAP (Extensible Authentication Protocol) mechanism.

 If the Windows VPN client could be used (with a custom EAP implementation) to connect to an AnyConnect server via IPsec/IKEv2, one could avoid installing a full VPN stack (parallel to the Windows stack) and also avoid the policies that are enforced by the AnyConnect client.

Unfortunately, there are some small but fatal obstacles to implementing this idea. This article describes where the problems lie.

IKEv2 Vendor ID Payload

RFC 7296, in section 3.12 says:

   The Vendor ID payload, denoted V in this document, contains a vendor-
   defined constant.  The constant is used by vendors to identify and
   recognize remote instances of their implementations.  This mechanism
   allows a vendor to experiment with new features while maintaining
   backward compatibility.

   A Vendor ID payload MAY announce that the sender is capable of

   accepting certain extensions to the protocol, or it MAY simply
   identify the implementation as an aid in debugging.

There does not seem to be any way to influence the Vendor ID payloads included in IKEv2 messages under Windows. Reading “[MS-IKEE]: Internet Key Exchange Protocol Extensions” seems to confirm this belief – the Microsoft IKEv2 implementation only sends particular Vendor ID payloads and not arbitrary payloads provided by an external component.

The IKE_SA_INIT message needs to include a Vendor ID payload with the value "CISCO-ANYCONNECT-EAP" in order to indicate that the client supports AnyConnect EAP, otherwise the Cisco VPN server will reject a subsequent attempt to use that EAP mechanism. A Cisco AnyConnect client also includes other Cisco Vendor ID payloads, but the absence of these payloads does not prevent an AnyConnect EAP connection from being established.

IKEv2 Identification Payload

The Cisco VPN server expects a particular (configurable) value for the Identification payload to be present in the first IKE_AUTH message; by default, the expected payload should contain an ID Type of 201 (the first “Private use” ID Type value) and the Identification Data "*$AnyConnectClient$*". If the client does not send the correct Identification payload, the connection attempt is aborted.

The Microsoft IKEv2 implementation does defer the contents of the Identification payload to an external component and calls out to the RasMan service, but there is no API to specify what RasMan should return. By default, it returns an ID_IPV4_ADDR ID Type but it can be persuaded (by setting the value “ExtendedIDiSupport” in the registry and meeting other requirements) to return an ID_FQDN ID Type. It is not possible to specify an arbitrary ID Type and Identification Data.

IKEv2 Fragmentation

The individual AnyConnect EAP messages can be large (especially if certificate authentication is used) and can (and do) easily exceed the size of the MTU for the path to the AnyConnect server. Windows only allows EAP messages to be up to MTU minus 100 bytes in size and suggests that a fragmentation mechanism be incorporated into the EAP mechanism if longer messages are needed.

AnyConnect Certificate Authentication

The mechanism used by AnyConnect to prove knowledge of the certificate private key depends on access to IKEv2 managed data (IKEv2 messages and derived keys), which is not available to EAP implementations under Windows.

 

IKEv2 Authentication Payload

The AnyConnect EAP mechanism is not key-generating, so the Authentication payloads should be computed (using the notation of RFC 7296) as:

AUTH = prf( prf(SK_pi, "Key Pad for IKEv2"), <InitiatorSignedOctets>)

AUTH = prf( prf(SK_pr, "Key Pad for IKEv2"), <ResponderSignedOctets>)

The Microsoft IKEv2 implementation seems to use a different computation for the Authentication payloads.

IKEv2 Extensible Authentication Protocol (EAP) Payload

Windows provides two frameworks to handle the EAP payloads in IKEv2 messages: the legacy EAP methods and the EAPHost Framework. The AnyConnect EAP method is not trivial but it is relatively straightforward and could easily be implemented using either framework.

Summary

With some serious hacking (which needs updating each time the relevant Microsoft DLL changes), it is possible to overcome each of these problems and the resulting VPN tunnel works well. It is however unlikely that this approach would ever be a viable option for connecting to an AnyConnect VPN server. The EAP fragmentation problem could be addressed be just letting the EAP method depend on IP fragmentation, if it chooses to do so. Mechanisms for setting IKEv2 Vendor Id and Identification payloads could be added to the Windows EAP frameworks. The Authentication payload problems looks like a bug that could be fixed. However the AnyConnect certificate authentication method seems to be irreconcilable with any reasonable compartmentalization of the IKEv2 and EAP implementations.

The IEapHostAuthenticatorSessionApis Interface

Windows 10 includes all necessary components bar one from being able to act as an EAP authenticator – it does not include the “Microsoft EAPHost Authenticator service” DLL (eapahost.dll). Fortunately, this DLL seems to have a simple task: to isolate the Windows VPN components from custom EAP implementations and to unify the two types of custom EAP implementations: legacy EAP methods (HKLM\SYSTEM\CurrentControlSet\Services\RasMan\PPP\EAP) and EapHost methods (HKLM\SYSTEM\CurrentControlSet\Services\Eaphost\Methods).

A separate article (Establishing a VPN connection from macOS to Windows 10) motivates why it might be useful to develop a replacement for this missing component. Essentially, the component implements just one public interface: IEapHostAuthenticatorSessionApis.

A C# definition of the interface, with parameter types chosen for convenience of implementation, is:

[ComImport, InterfaceType(ComInterfaceType.InterfaceIsIUnknown), Guid("5A8371A3-0C6D-487B-B3C8-46D785C4C940")]
interface IEapHostAuthenticatorSessionApis
{
    void BeginSession(uint flags, EAP_HOST_AUTHENTICATOR_METHOD_DATA_ARRAY methods, EAP_ATTRIBUTES ea, uint maxpack, [MarshalAs(UnmanagedType.LPWStr)] string identity, out uint session, out EAP_ERROR error);
    void UpdateInnerMethodParams(uint session, uint flags, [MarshalAs(UnmanagedType.LPWStr)] string identity, EAP_ATTRIBUTES ea, out EAP_ERROR error);
    void IsReplay(uint session,  uint n, EapPacket packet, [MarshalAs(UnmanagedType.Bool)] out bool replay, out EAP_ERROR error);
    void ReceivePacket(uint session, uint n, IntPtr p, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);
    void SendPacket(uint session, out uint n, out IntPtr p, out EAP_AUTHENTICATOR_SEND_TIMEOUT timeout, out EAP_ERROR error);
    void GetAttributes(uint session, [Out] EAP_ATTRIBUTES ea, out EAP_ERROR error);
    void SetAttributes(uint session, EAP_ATTRIBUTES ea, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);
    void SetAuthenticateResult(uint session, uint status, [Out] EAP_ATTRIBUTES ea, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);
    void GetResult(uint session, [Out] EAP_METHOD_AUTHENTICATOR_RESULT2 result, out EAP_ERROR error);
    void GetFinalPacket(uint session, [MarshalAs(UnmanagedType.Bool)] bool success, out uint n, out IntPtr p, out EAP_ERROR error);
    void EndSession(uint session, out EAP_ERROR error);
}

An ad-hoc “journal” (explained in detail later) of the methods called and interesting input/output arguments during a PEAP authentication is:

BeginSession(flags = 0, maxpack = 1396, identity = CHBSGRN02081961\gary)
  Method 0 26
  Method 1 13
  Method 2 25
  EA 0 eatFramedMTU 4 78-05-00-00
  EA 1 eatAcctSessionId 2 34-34
  EA 2 eatNASIdentifier 15 43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31
  EA 3 eatNASIPAddress 4 C0-A8-00-02
  EA 4 eatServiceType 4 02-00-00-00
  EA 5 eatFramedProtocol 4 01-00-00-00
  EA 6 eatNASPort 4 04-00-00-00
  EA 7 eatNASPortType 4 05-00-00-00
  EA 8 eatTunnelType 4 03-00-00-00
  EA 9 eatTunnelMediumType 4 01-00-00-00
  EA 10 eatCalledStationId 11 31-39-32-2E-31-36-38-2E-30-2E-32
  EA 11 eatTunnelServerEndpoint 11 31-39-32-2E-31-36-38-2E-30-2E-32
  EA 12 eatCallingStationId 11 31-39-32-2E-31-36-38-2E-30-2E-35
  EA 13 eatTunnelClientEndpoint 11 31-39-32-2E-31-36-38-2E-30-2E-35
  EA 14 eatEAPMessage 9 02-00-00-09-01-67-61-72-79
  EA 15 eatUserName 4 67-61-72-79
  EA 16 4108 4 C0-A8-00-02
  EA 17 4128 15 43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31
  EA 18 8132 4 02-00-00-00
  EA 19 4147 4 37-01-00-00
  EA 20 4148 10 4D-53-52-41-53-56-35-2E-32-30
  EA 21 4160 11 4D-53-52-41-53-56-35-2E-32-30-00
  EA 22 4159 13 4D-53-52-41-53-2D-30-2D-47-41-52-59-00
  EA 23 8158 39 7B-45-41-41-36-38-39-43-44-2D-38-46-45-41-2D-34-33-30-44-2D-39-35-44-37-2D-31-44-38-46-38-37-41-37-31-44-39-31-7D-00
ReceivePacket(1B1DA2F0, n = 9)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
EndSession(1B147190)
IsReplay(1B1DA2F0, n = 166)
  2 1 166
ReceivePacket(1B1DA2F0, n = 166)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 6)
  2 2 6
ReceivePacket(1B1DA2F0, n = 6)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 110)
  2 3 110
ReceivePacket(1B1DA2F0, n = 110)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 6)
  2 4 6
ReceivePacket(1B1DA2F0, n = 6)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 40)
  2 5 40
ReceivePacket(1B1DA2F0, n = 40)
  action = Send -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 51)
  2 6 51
ReceivePacket(1B1DA2F0, n = 51)
  action = SendWithTimeout -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 94)
  2 7 94
ReceivePacket(1B1DA2F0, n = 94)
  action = SendWithTimeout -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 37)
  2 8 37
ReceivePacket(1B1DA2F0, n = 37)
  action = IndicateTLV -> EAP_METHOD_AUTHENTICATOR_RESPONSE_RESPOND, code = 0
GetAttributes(1B1DA2F0)
  EA 0 eatVendorSpecific 22 00-00-01-37-0A-12-01-43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31
  EA 1 eatVendorSpecific 49 00-00-01-37-1A-2D-01-53-3D-33-39-46-34-46-32-35-44-44-34-33-34-45-37-41-46-31-43-46-31-37-38-42-33-41-31-36-45-45-32-38-39-30-41-43-39-35-30-43-36
  EA 2 eatEAPTLV 6 80-03-00-02-00-01
  EA 3 eatEAPTLV 16 00-07-00-0C-00-00-01-37-00-05-00-04-00-00-00-1A
SetAttributes(1B1DA2F0)
  EA 0 eatEAPTLV 6 80-03-00-02-00-01
ReceivePacket(1B1DA2F0, n = 0)
  action = SendWithTimeoutInteractive -> EAP_METHOD_AUTHENTICATOR_RESPONSE_SEND, code = 0
SendPacket(1B1DA2F0)
IsReplay(1B1DA2F0, n = 106)
  2 10 106
ReceivePacket(1B1DA2F0, n = 106)
  action = SendAndDone -> EAP_METHOD_AUTHENTICATOR_RESPONSE_RESULT, code = 0
GetResult(1B1DA2F0)
  EA 0 eatVendorSpecific 22 00-00-01-37-0A-12-01-43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31
  EA 1 eatVendorSpecific 49 00-00-01-37-1A-2D-01-53-3D-33-39-46-34-46-32-35-44-44-34-33-34-45-37-41-46-31-43-46-31-37-38-42-33-41-31-36-45-45-32-38-39-30-41-43-39-35-30-43-36
  EA 2 eatPEAPFastRoamedSession 4 00-00-00-00
  EA 3 eatPEAPEmbeddedEAPTypeId 4 1A-00-00-00
  EA 4 eatVendorSpecific 56 00-00-01-37-10-34-00-00-20-BC-B2-F4-FB-96-F8-B0-D1-C5-68-C2-9E-DE-02-DF-9B-11-C1-AA-3E-A0-FF-47-68-D4-FC-F1-92-A8-9B-AE-03-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
  EA 5 eatVendorSpecific 56 00-00-01-37-11-34-00-00-20-9C-07-6C-93-D0-DC-4F-41-57-97-86-03-6C-4C-56-2A-AA-BD-BD-8D-71-3E-3A-B2-BE-F7-78-B6-2F-92-60-B9-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

GetFinalPacket(1B1DA2F0, success = True)

BeginSession

Each authenticator session starts with a call to BeginSession.

Most of the arguments to BeginSession just need to be saved for later use: the set of EAP_FLAG_* values, the set of EAP_ATTRIBUTE values, the maximum size of an EAP packet and the remote identity (an “unauthenticated” identity, created using information from things like the IKEv2 Identification payload and the initial EAP Identity Response). 

The argument containing an array of potential EAP methods needs to be acted upon immediately. This list is presumably constructed by Windows based on the installed EAP methods and the policy settings. A method must be chosen from this list and the location (path to DLL) and type of the EAP method looked up. The DLL is then loaded and, according to its type (legacy or EapHost), the appropriate “initialize” and “begin” methods are called.

The EAP_ATTRIBUTE values are Type/Length/Value (TLV) items, the “Type” indicating the meaning of the value rather than a simple data type (e.g. string, integer), so without knowing the format (or underlying data type) for each “Type” it is easiest to “print” the values as a sequence of bytes. For those types that are in the EAP_ATTRIBUTE_TYPE enumeration, the name is printed. The other type values can be looked up in the sdoias.idl file in the Windows Software Development Kit. For the example above, the meanings of the numeric “Types” are:

4108 IAS_ATTRIBUTE_CLIENT_IP_ADDRESS
4128 IAS_ATTRIBUTE_CLIENT_NAME
8132 MS_ATTRIBUTE_NETWORK_ACCESS_SERVER_TYPE
4147 MS_ATTRIBUTE_RAS_VENDOR
4148 MS_ATTRIBUTE_RAS_VERSION
4160 MS_ATTRIBUTE_RAS_CLIENT_VERSION
4159 MS_ATTRIBUTE_RAS_CLIENT_NAME
8158 MS_ATTRIBUTE_RAS_CORRELATION_ID

For interest only (there is no need to act on the information), one of the EAP_ATTRIBUTE values is the EAP Identity Response:

  EA 14 eatEAPMessage 9 02-00-00-09-01-67-61-72-79

  02 = Code (Response)
  00 = Identifier (0)
  00-09 = Length (9)
  01 = Type (Identity)
  67-61-72-79 = Data (“gary”)

ReceivePacket and SendPacket

The authentication process is advanced by the VPN server making the newly received EAP packet available via a call to ReceivePacket. ReceivePacket must then invoke the EAP method with this input; the EAP method typically returns a new EAP packet to be sent back to the VPN client and an indication of what to do, such as “send with a timeout” or “send with an interactive timeout” (a longer timeout that allows for the possibility that the VPN client might need to interact with the user). The “ready to be sent” EAP packet is cached until the VPN server calls SendPacket to retrieve it.

IsReplay

The VPN server typically calls the IsReplay packet with an EAP packet before making the packet available by another call to ReceivePacket. The EAP methods don’t have any entry point that specifically checks for a replayed packet, so the simplest thing to do is to state that the EAP packet is not a replay and just let the authentication fail if that was a mistake.

GetResult

At some point (after a number of exchanges of EAP packets), the EAP method will indicate that an authentication result has been reached. The VPN server calls GetResult to retrieve the result (including a success/failure indication, a failure reason in the event of a failure and a set of EAP_ATTRIBUTE values derived during the authentication process.

GetFinalPacket

The VPN server finally calls GetFinalPacket to retrieve the packet that will communicate the success/failure of the authentication back to the VPN client.

GetAttributes and SetAttributes

For a simple authentication, such as EAP-MSCHAPv2, GetAttributes and SetAttributes are not needed/called, but the example authentication shown above is a more complex scenario (PEAP) and these two routines bridge the gap between completing the first phase of the authentication (building the protected (TLS) channel) and starting the second phase (MSCHAPv2).

GetAttributes makes a set of EAP_ATTRIBUTE values derived during the first phase of authentication process available to the VPN server and SetAttributes makes a new set of EAP_ATTRIBUTE values available to the next phase of the authentication process.

EndSession

EndSession is called when the authentication session is no longer needed, prior to “releasing” the COM object.

UpdateInnerMethodParams and SetAuthenticateResult

I have not encountered any scenarios where these are called.

eatVendorSpecific and eatEAPTLV TLVs

An IEapHostAuthenticatorSessionApis implementation does not need to understand or act upon the EAP_ATTRIBUTE values that flow through it, but it might be interesting to look in more detail at the contents of the eatVendorSpecific and eatEAPTLV TLVs.

eatVendorSpecific 22 00-00-01-37-0A-12-01-43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31

00-00-01-37 = Vendor-Id (311 (Microsoft) in big-endian format)
0A = Vendor-Type (MS-CHAP-Domain)
12 = Vendor-Length (0x12)
01 = Ident
43-48-42-53-47-52-4E-30-32-30-38-31-39-36-31 = String (CHBSGRN02081961)

eatVendorSpecific 49 00-00-01-37-1A-2D-01-53-3D-33-39-46-34-46-32-35-44-44-34-33-34-45-37-41-46-31-43-46-31-37-38-42-33-41-31-36-45-45-32-38-39-30-41-43-39-35-30-43-36

00-00-01-37 = Vendor-Id (311 (Microsoft) in big-endian format)
1A = Vendor-Type (MS-CHAP2-Success)
2D = Vendor-Length (0x2D)
01 = Ident
53-3D-… = String (S=39F4F25DD434E7AF1CF178B3A16EE2890AC950C6)

eatVendorSpecific 56 00-00-01-37-10-34-00-00-20-BC-B2-F4-FB-96-F8-B0-D1-C5-68-C2-9E-DE-02-DF-9B-11-C1-AA-3E-A0-FF-47-68-D4-FC-F1-92-A8-9B-AE-03-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

00-00-01-37 = Vendor-Id (311 (Microsoft) in big-endian format)
10 = Vendor-Type (MS-MPPE-Send-Key)
34 = Vendor-Length (0x34)
00-00 = Salt
20 = Key-Length
BC-B2-… = Key
00-00-… = Padding

eatVendorSpecific 56 00-00-01-37-11-34-00-00-20-9C-07-6C-93-D0-DC-4F-41-57-97-86-03-6C-4C-56-2A-AA-BD-BD-8D-71-3E-3A-B2-BE-F7-78-B6-2F-92-60-B9-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00

00-00-01-37 = Vendor-Id (311 (Microsoft) in big-endian format)
11 = Vendor-Type (MS-MPPE-Recv-Key)
34 = Vendor-Length (0x34)
00-00 = Salt
20 = Key-Length
9C-07-… = Key
00-00-… = Padding

eatEAPTLV 6 80-03-00-02-00-01

80-03 = Type (Result TLV (mandatory))
00-02 = Length
00-01 = Value (Success)

eatEAPTLV 16 00-07-00-0C-00-00-01-37-00-05-00-04-00-00-00-1A

00-07 = Type (Vendor Specific TLV)
00-0C = Length
00-00-01-37 = Vendor-Id (311 (Microsoft) in big-endian format)
00-05 = Vendor-Type (5)
00-04 = Length
00-00-00-1A = Value (EAP-MSCHAPv2)

Establishing a VPN connection from macOS to Windows 10

Windows 10 (i.e. a non-server version of Windows) can act as a VPN server. Searching the Internet for “Windows 10” coupled with “VPN server” or “incoming connection” will probably turn up a few guides on how to set up Windows 10 as a VPN server. However, the difficulties of matching VPN client and server capabilities in terms of VPN transport and authentication protocols are often not mentioned. This article looks at the concrete case of establishing a VPN connection from a macOS client to a Windows 10 server using built-in VPN functionality (i.e. not using third party VPN products on either system).

The version of macOS (High Sierra, 10.13.6) used for testing purposes offers 3 “types” of VPN:

1.       L2TP over IPSec

2.       Cisco IPSec

3.       IKEv2

Windows 10 offers 4 types:

1.       PPTP

2.       L2TP/IPsec

3.       SSTP

4.       IKEv2

Cisco IPsec uses “Extended Authentication within IKE (XAUTH)” (for which there is a draft RFC from 2001: draft-beaulieu-ike-xauth-02.txt). Windows rejects connection attempts that try to negotiate these extensions.

SSL VPN (VPN over SSL) clients are available for macOS from the Apple Store, but a cursory examination did not find any that claim compatibility with Microsoft SSTP.

Using L2TP over IPsec

The transport protocol (L2TP in an IPsec Encapsulating Security Payload (ESP) tunnel) is common to both systems (Windows 10 and macOS).

There are two macOS options for “Machine Authentication” (IKEv1 Phase 1 authentication):

1.       Shared Secret

2.       Certificate

Shared Secret

“Shared Secret” seems easiest for a simple home network set-up, but the Windows “Network Connections” control panel applet does not provide any “user interface” for setting a shared secret (this applies to the user interfaces for the VPN server). There is however a trick that seems to work: define a “dummy” IPsec tunnel with “Preshared key” authentication; any tool can be used to do this (e.g. the “Windows Defender Firewall with Advanced Security” control panel applet, the “netsh advfirewall consec add rule” command or the “New-NetIPsecRule” PowerShell cmdlet).

Here are examples of creating suitable “dummy” IPsec tunnels (the endpoint value can be any address that does not apply to any potential traffic) with a shared secret of “XXX”:

netsh advfirewall consec add rule name="Dummy" endpoint1=192.168.3.1/32 endpoint2=any mode=tunnel action=requireinrequireout auth1=computerpsk auth1psk="XXX"

New-NetIPsecRule -DisplayName "Dummy" -Phase1AuthSet (New-NetIPsecPhase1AuthSet -DisplayName "Dummy" -Proposal (New-NetIPsecAuthProposal -Machine -PreSharedKey "XXX")).InstanceID -InboundSecurity Require -OutboundSecurity Require -KeyModule IKEv1 -Mode Tunnel -LocalAddress 192.168.3.1/32 -RemoteAddress Any

Obviously, having more than one IPsec tunnel definition with a (different) pre-shared key is problematic.

Certificate

The “Certificate” option is also not straightforward. By default, the registry value of “ServerFlags” (documented in section 2.2.3.4.6 of [MS-RRASM]: Routing and Remote Access Server (RRAS) Management Protocol) under the key “HKLM\SYSTEM\CurrentControlSet\Services\RemoteAccess\Parameters” does not include the option “Authentication using certificates is allowed on the RRAS server” (0x04000000), so this must be explicitly added.

One also needs to create and manage a small certificate authority. I initially made a mistake by omitting the text highlighted below when creating a “root authority” certificate:

New-SelfSignedCertificate -CertStoreLocation Cert:\CurrentUser\My -KeyAlgorithm RSA -KeyLength 4096 -KeyExportPolicy Exportable -KeyUsage CertSign -NotAfter 2061-08-01 -Subject "CN=insert your name here" -Type Custom -TextExtension "2.5.29.19={text}CA=True"

The absence of the highlighted text did not affect the efficacy of the “root” certificate under Windows but caused “Trust evaluate failure: [root AnchorTrusted BasicConstraints]” under macOS.

When configuring the L2TP VPN under macOS, there is a field named “Server Address”; for the “Shared Secret” option, a dotted notation IPv4 address can be used but for the “Certificate” option one has to use a name (matching the certificate’s subject common name or DNS subject alternative name) and add the name and address to /etc/hosts.

Phase 2 (User) Authentication

macOS offers 5 possibilities (Password, RSA SecurID, Certificate, Kerberos, CryptoCard), only one of which I bothered to pursue: “Password” – this choice results in MSCHAPv2 authentication taking place.

Summary

L2TP was documented as RFC 2661 in 1999 – decidedly older than the other macOS option of IKEv2 (which was documented as RFC 5996 in 2010 and obsoleted by RFC 7296 in 2014). The transport protocol suffers from quite a high degree of encapsulation – below is a view (taken from Microsoft’s Message Analyzer) after the IPsec ESP encapsulation has been removed (replaced by “IP in IP” encapsulation after decryption – shown as Message2V4 by Message Analyzer):

Using IKEv2

When configuring the IKEv2 VPN under macOS, there are fields named “Server Address” and “Remote ID”; in contrast to the L2TP VPN, one has to use a dotted notation IPv4 address for the “Server Address” because it seemed as though macOS only tries to resolve the name via DNS (and not via /etc/hosts). The “Remote ID” field needs a name matching the VPN server certificate.

The “Authentication Settings” for IKEv2 are structured differently from L2TP; macOS offers: Username, Certificate, None.

The name “None” is rather misleading – one can interpret it as “do not use EAP” (i.e. include an AUTH payload from the first message in the IKE_AUTH exchange – the absence of such a payload indicates that EAP will be used instead). In this case both server and client must possess and use certificates to authenticate. The difficulties of using EAP (coding a replacement for a component not included in Windows 10) makes “None” a good choice of authentication mechanism.

EAP

“Certificate” authentication means authenticate with EAP-TLS and “Username” authentication means authenticate with EAP-MSCHAPv2 or PEAP.

Both authentication methods need to use EAP, but this is also disabled by default in the Windows RemoteAccess “ServerFlags” setting. One needs to explicitly set “EAP protocol can be negotiated for remote access and demand dial connection authentication” (0x00008000).

The file %SystemRoot%\System32\ias\ias.xml seems to be a replacement for the policy database of a full NPS (Network Policy Server) and some changes need to be made there too. In the section “Connections_to_Microsoft_Routing_and_Remote_Access_server”, the collection of “msNPAuthenticationType2” elements needs to be expanded to include “5” (IAS_AUTH_EAP) and optionally “11” (IAS_AUTH_PEAP); if IAS_AUTH_PEAP is added (and one wants to use PEAP) then the collection of “msNPAllowedEapType” elements needs to be expanded to include “19000000000000000000000000000000”.

Missing EAP Component

Windows 10 does not include the “Microsoft EAPHost Authenticator service” DLL (eapahost.dll) although it does include many (all?) other EAP components found on Windows Server. Fortunately, this DLL seems to have a simple task: to isolate the Windows VPN components from custom EAP implementations and to unify the two types of custom EAP implementations: legacy EAP methods (HKLM\SYSTEM\CurrentControlSet\Services\RasMan\PPP\EAP) and EapHost methods (HKLM\SYSTEM\CurrentControlSet\Services\Eaphost\Methods).

One needs to implement the interface below (described in a separate article) and package the result as a COM out-of-process server:

[ComImport, InterfaceType(ComInterfaceType.InterfaceIsIUnknown), Guid("5A8371A3-0C6D-487B-B3C8-46D785C4C940")]

interface IEapHostAuthenticatorSessionApis

{

    void BeginSession(uint flags, EAP_HOST_AUTHENTICATOR_METHOD_DATA_ARRAY methods, EAP_ATTRIBUTES ea, uint maxpack, [MarshalAs(UnmanagedType.LPWStr)] string identity, out uint session, out EAP_ERROR error);

    void UpdateInnerMethodParams(uint session, uint flags, [MarshalAs(UnmanagedType.LPWStr)] string identity, EAP_ATTRIBUTES ea, out EAP_ERROR error);

    void IsReplay(uint session,  uint n, EapPacket packet, [MarshalAs(UnmanagedType.Bool)] out bool replay, out EAP_ERROR error);

    void ReceivePacket(uint session, uint n, IntPtr p, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);

    void SendPacket(uint session, out uint n, out IntPtr p, out EAP_AUTHENTICATOR_SEND_TIMEOUT timeout, out EAP_ERROR error);

    void GetAttributes(uint session, [Out] EAP_ATTRIBUTES ea, out EAP_ERROR error);

    void SetAttributes(uint session, EAP_ATTRIBUTES ea, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);

    void SetAuthenticateResult(uint session, uint status, [Out] EAP_ATTRIBUTES ea, out EAP_METHOD_AUTHENTICATOR_RESPONSE_ACTION action, out EAP_ERROR error);

    void GetResult(uint session, [Out] EAP_METHOD_AUTHENTICATOR_RESULT2 result, out EAP_ERROR error);

    void GetFinalPacket(uint session, [MarshalAs(UnmanagedType.Bool)] bool success, out uint n, out IntPtr p, out EAP_ERROR error);

    void EndSession(uint session, out EAP_ERROR error);

}

A “proof-of-concept” implementation (including all necessary C# structure and enumeration definitions) occupied about 500 lines of code (quite small), but it violates the “using only built-in functionality” goal. Using the “None” authentication method is the way to meet that goal.

IKEv2 Policy Settings

The default policy settings are documented in section 2.2.3.4.2.8 of “[MS-RRASM]: Routing and Remote Access Server (RRAS) Management Protocol”. In summary, they are:

IntegrityMethod	INTEGRITY_SHA_256
EncryptionMethod	CIPHER_AES_256
CipherTransformConstant	CIPHER_CONFIG_CBC_3DES
AuthTransformConstant	AUTH_CONFIG_HMAC_SHA_256_128
PfsGroup	PFS_2048
DHGroup	DH_GROUP_2

The use of Triple DES for the Quick Mode (QM) cipher might be considered weak – as might the use of Diffie-Hellman (DH) Group 2. With these defaults, macOS uses DH Group 14 in its first IKE_SA_INIT exchange, only to have that rejected when Windows replies with a Notify payload of type INVALID_KE_PAYLOAD and macOS falls back to DH Group 2. A stronger policy can be configured, using the information in [MS-RRASM].

Summary

The view below (taken from Microsoft’s Message Analyzer) after IPsec ESP encapsulation has been removed shows how little encapsulation takes place compared to an L2TP VPN:

Addressing, Routing and Firewall

The only interesting setting option on the Windows 10 “Incoming Connections” object is the “IP address assignment”. The option “Assign IP addresses automatically using DHCP” does work, even if there is no DHCP service – if there is no DHCP service, the VPN server allocates addresses itself from the 169.xxx.xxx.xxx range; however a VPN connection attempt might fail with ERROR_IPSEC_IKE_INNER_IP_ASSIGNMENT_FAILURE before it is determined that no DHCP service is available. I chose “Specify IP addresses” and used a sub-range of the local network.

Although Windows 10 will forward IP traffic, the Windows 10 VPN server does nothing to advertise routes. The Windows 10 VPN server will however respond appropriately to ARP requests for its VPN clients.

By default, the VPN network will be assigned to the “Public” firewall profile (which, by default, blocks access to many services). After changing the profile to “Private”, packets were still dropped by a blocking WFP filter with a FWPM_CONDITION_ORIGINAL_PROFILE_ID condition unless the requested service was accessible via the “Public” profile – this is just an observation (the cause and resolution have not yet been determined).