API Management - Networking FAQs (Demystifying Series II) (2024)

This is a continuation of demystifying common issues faced while integrating API Management instance in a virtual network. You can find part-I here.

FAQ Scenarios

• APIM with Azure Firewall
• Using custom DNS with APIM
• Force tunneling traffic from Azure to on-premises over ExpressRoute or VPN

APIM with Azure Firewall

API Management instance can be configured to run in a VNET internal or external mode. Azure Firewall can be used to control and monitor the APIM subnet traffic.

Can you run APIM in VNET external mode with user-defined route forcing default traffic to Azure firewall?

No. This configuration will not work as client will try to access API Management Gateway/proxy on its public IP address but the response from API Management Gateway will be forwarded to Azure Firewall.

Azure Firewall being fully-stateful will drop the response traffic.

Scenario: Forcing APIM subnet traffic through Azure Firewall using user-defined routes.

For monitoring purposes, we create user-defined routes to point default traffic (0.0.0.0/0) from API subnet to Azure Firewall.

Route table having user-defined route (named FW) pointing default traffic (denoted with address prefix as 0.0.0.0/0) to the next hop as Azure Firewall.

With the above rule, management traffic (managing the APIM configuration using Azure portal or PowerShell) response is also forced through Azure Firewall.

With the above configuration in place, you observe that API are not loaded and keep on loading the page. If you inspect the network trace, you will observe the Management endpoint not reachable error.

Error message in the developer console

Failed to connect to management endpoint servicename.management.azure-api.net:3443 for a service deployed in a Virtual Network. Make sure to follow guidance at https://aka.ms/apim-vnet-common-issues.

Solution:

We must route the management endpoint response traffic directly to internet to avoid response traffic getting dropped by Azure Firewall.

This can be achieved by adding routes in the route table associated with APIM subnet.

We must add user defined routes for the control plane IP address with next hop as internet. The above image shows routes for global control plane IP address and region-specific IP address. You can find the control plane IP address for the specific region in https://docs.microsoft.com/en-us/azure/api-management/api-management-using-with-vnet#--control-plane...

Using custom DNS with APIM

Can I use azure default DNS server for APIM?

Yes, you can use azure default DNS server in the following conditions:

• if your solution/architecture does not require connectivity to on-premises or you are not accessing on-premises resources using hostnames.
• if all your backend servers are hosted in azure environment.

What is the limitation of default azure DNS?

When creating a Virtual Network, Azure provides default DNS server "168.63.129.16". You can check this under Virtual Network-> DNS Severs blade. This DNS server is used for name resolution for azure services but it cannot provide name resolution for on-premise resources.

When do we need custom DNS for APIM VNET?

When APIM is running in internal or external VNET mode and your backend APIs are hosted in on-prem servers, you establish private connectivity to the on-premise network using ExpressRoute or Site to Site VPN.

The on-prem servers will host the APIs. Although you can enable communications by using IP addresses, it is much simpler to use names that can be easily remembered, and do not change.

Azure default DNS server cannot resolve on-prem host names. So, we add and manage custom DNS servers for name resolution purposes. We must also take steps to resolve the Azure services as well by using DNS forwarder. Custom DNS should resolve on-premises resources and forwarder will resolve the azure resources.

Please note that forwarding to "168.63.129.16" will not work outside Azure i.e if your custom DNS server is in on-premises, you cannot forward queries to "168.63.129.16" to resolve Azure services.

An example DNS forwarder is available in theAzure Quickstart Templates galleryandGitHub

Example:

Why network connectivity show failures with custom DNS server?

APIM has dependencies on other resources like SQL and storage. When custom DNS server is configured and if it cannot resolve the Azure services dependency, you will notice network connectivity failures.

Example: Failed to connect to https://westeurope.prod.warm.ingestion.msftcloudes.com with error The remote name could not be resolved: 'westeurope.prod.warm.ingestion.msftcloudes.com'

This is because of custom DNS is not resolving or forwarding DNS queries to azure. We need to manage the DNS records or forward DNS queries as mentioned here https://docs.microsoft.com/en-us/azure/virtual-network/virtual-networks-name-resolution-for-vms-and-...

How can I check if my custom DNS server is resolving azure services?

You can use nslookup command to test if the custom DNS server is able to resolve the Azure services. RDP into any virtual machine in virtual network and test the nslookup command for the Azure PaaS services or on-premises hostnames. You can find the name of the azure services in Network connectivity status blade.

Force tunneling traffic from Azure to on-premises over ExpressRoute or VPN

When APIM is running in VNET external or internal mode, we often see virtual network is connected to on-premises network using Site-to-Site VPN or ExpressRoute.

What is the significance of force tunneling?

Force tunneling lets you redirect or "force" all Internet-bound traffic back to your on-premises location via a Site-to-Site VPN tunnel for inspection and auditing. This is a critical security requirement for most enterprise IT policies. Without force tunneling, Internet-bound traffic from your APIM Vnet will traverse from Azure network infrastructure directly out to the Internet, without the option to allow you to inspect or audit the traffic. Unauthorized Internet access can potentially lead to information disclosure or other types of security breaches.

Ways to do force tunneling:

1. Force tunneling over Site-to-Site VPN (without BGP - Border Gateway Protocol ) Force tunneling in Azure is configured via virtual network user-defined routes pointing default traffic to the next hop as Virtual Network Gateway. Post this, you need to set a "default site" among the cross-premises local sites connected to the virtual network. Refer here: https://docs.microsoft.com/en-us/azure/vpn-gateway/vpn-gateway-forced-tunneling-rm#:~:text=Forced%20...

Force tunneling can also be configured on Site-to-Site VPN tunnel with BGP (commonly called as BGP over IPsec) where default route is advertised by on-premises to Azure over BGP sessions.

2. Force tunneling over ExpressRoute: Force tunneling is enabled by advertising a default route via the ExpressRoute BGP peering sessions. To know more about asymmetric route with ExpressRoute -https://docs.microsoft.com/en-us/azure/expressroute/expressroute-asymmetric-routing#asymmetric-routi...

What is effect of force tunneling APIM traffic?

With force tunneling, default/Internet traffic from APIM subnet is forced to flow through on-premises which could lead to below possibilities:

• On-premises firewall might block internet traffic and connectivity to APIM instance is lost.
• APIs will never load from portal since the Resource provider will access Management endpoint on its public IP addresses but the response is force tunneled to on-premises which will be blocked by fully-stateful firewalls.

What is asymmetric route?

In Asymmetric routing, a packet traverses from a source to a destination in one path and takes a different path when it returns to the source.

What is the solution for asymmetric route?

We must avoid routing outgoing response from API Management subnet to on-premises. To achieve this, we must add precise user defined route to force the traffic through internet.

The APIM Resource Provider uses a set of static IP addresses called as control plane IP addresses. You can find control plane Ip addresses inhttps://docs.microsoft.com/en-us/azure/api-management/virtual-network-reference?tabs=stv2#control-pl...

List of control plane IP address to be added in route table,

Regardless of the region your APIM instance is running, you must add global control plane IP address.

Find the IP address of the specific region your APIM instance is running and add.

Finally, the UDR will have the following routes as shown in the below image.

Why APIM reporting Network Connectivity Status issues after adding UDR to force traffic to on-premises firewall?

Since APIM subnet traffic is routed to on-premises firewall and it blocks the connectivity to azure services. You would notice warnings and errors under Network Connectivity Status blade . This network connectivity issues will cause degradation in the ability to use monitor the API Management service.

The below images showing issues with such dependencies.

To resolve the network connectivity issues, we must whitelist these services in on-premises firewall or you can add additional precise routes in the route table.

For Azure Monitor we can add IP address prefix 13.92.40.223/32 with next hop as Internet as shown below.