Azure ILB ASE v2 Advanced Scenarios (PCI DSS, Application Gateway, CI/CD, Multisite hosting, SSL and more)
by Chenlong 2019-03-06
language of the article
5120 4 TECH azure ilb-ase application-gateway
Intro
When dealing with the isolated hosting environment in Azure (known as ASE), devs could come across complicated subjects, but unfortunately, the official documentation doesn't help much: https://github.com/MicrosoftDocs/azure-docs/issues/10588
Knowing that ASE is a pretty advanced feature (pricy as well) in Azure, many enterprises choose it because of security reasons. One of the typical scenarios is the PCI DSS architecture as described on this page: https://docs.microsoft.com/en-us/azure/security/blueprints/pcidss-paaswa-overview
If you're dealing with the payment systems, you want to protect your services as much as possible. However, setting this whole infrastructure up can really be a handful, especially for beginners.
So, after weeks of searching and testing, I managed to provision the PCI DSS infrastructure in Azure with ILB ASE (App Service Environment v2 with Internal Load Balancer) behind the Application Gateway (WAF), that hosts multiple applications of TLS/SSL, with CI/CD pipeline enabled in Azure DevOps.
I hope this article could be helpful and eventually saves your time. Now let's cover the basics in order.
1. Add Virtual Network and Subnets
Add a new VNet in your subscription, configure subnets for ILB ASE, WAF App Gateway, Virtual Machine which will be used as both DNS server and build agent later.
2. Create ASE v2 with ILB as VIP Type
2.1 For ASE created before May 2019
Now create your ASE v2 from the portal, make sure that you select the virtual network created in step 1, choose the "ilb" subnet and the Internal VIP Type, as for the Domain, use something unique as suggested by MS, ex. internal-mydomain.com if your services' domain name is mydomain.com.
Here the ownership of the domain name internal-mydomain.com is not required, as it will be used internally only.
The creation of the ASE could take a couple of hours, so just relax and take a break :)
Once it's deployed, you need to upload the ILB certificate, if you own the domain you can purchase an app certificate and use it directly, otherwise, you can also generate a self-signed certificate as described in the official documentation: https://docs.microsoft.com/en-us/azure/app-service/environment/create-ilb-ase#post-ilb-ase-creation-validation
Make sure it covers both the *.internal-mydomain.com and the *.scm.internal-mydomain.com
2.2 For ASE created after May 2019
You don't need to generate certificate anymore and ASE will have the default appserviceenvironment.net domain name.
3. Add Service Plan and Apps in your ASE
This step is pretty simple, you only need to make sure to create your service plan inside the ASE that you just added in step 2, and here in my example I have 4 Web App + 3 Slots, you should add apps to your ASE according to your own needs.
4. Create Application Gateway
Add new application gateway from Azure Portal, choose the WAF tier so that you'll have the application firewall feature, choose the VNet created in step 1 and also the waf subnet, create a new public IP address and give it a friendly DNS name such as appgw-mydomain.westeurope.cloudapp.azure.com, configure the listener to be HTTPS, then confirm the creation.
5. Configure Application Gateway
Here it comes the most important section! After the creation of your appgw, you need to configure it for multisite hosting (HTTPS), and I recommend you to follow the below steps:
5.1 Backend pool
Put the Internal Load Balancer IP address of your ILB ASE in the target of type IP address or FQDN, you can find it as shown here:
5.2 DNS names of your apps
Log in to your DNS provider portal, and add the CNAME records for your applications, such as
app-1.mydomain.com
app-2.mydomain.com
...
Point them all to your newly created application gateway DNS appgw-mydomain.westeurope.cloudapp.azure.com.
5.3 Health probes
Add new health probes for each of your app, select HTTPS and put the registered names of your app as Host
5.4 Http settings
Add http setting for each of the apps, select HTTPS as protocol, and upload the .CER certificate of your self-signed certificate of internal-mydomain.com
You can use 3rd party tools to export your certificate as .CER file such as DigiCert Utility
If you've purchased the Azure App Certificate for internal-mydomain.com, you may find this article very useful to manage the certificate locally: https://azure.github.io/AppService/2017/02/24/Creating-a-local-PFX-copy-of-App-Service-Certificate.html
Remember to select the custom probe that you've added in step 5.3
Latest update: For ASE created after May 2019, you need to get the .CER for the default appserviceenvironment.net domain, I found the official doc's description is a bit rough, however, a useful tip here for you is to add a VM inside the same VNET of the ASE, and navigate to a simple ASE hosted application, then you can export the website certificate in base 64 encoded .CER from your browser.
5.5 Listeners
You need to add Multi-site listener for your apps in this scenario, choose a pertinent name and select the default front IP configuration (which points to the public IP of your application gateway), and the port 443 for HTTPS protocol. Here you need to type the same hostname of your app as you did in step 5.3
5.6 Rules
Add Basic rule for each of your apps. You need to simply select the corresponded Listener and HTTP setting of the app from the dropdown list.
5.7 Application Custom Domain
If you managed to make it this far, congratulations!
This is the last configuration you need to do about the Application Gateway. Remember that all your apps are hosted inside the private ASE now and they're not accessible from the public network. Now you need to configure their custom domain so that they'll have the same hostname as you configured in your http listeners.
Navigate to the App Service, go to Settings -> Custom domains section, and add the app-1.mydomain.com etc. to your app. Once added, you also need to bind the SSL with your private certificate (.pfx) of your domain mydomain.com
5.8 Final test
Test your application gateway by simply navigate to https://app-1.mydomain.com
Check in the HTTP headers if your request hits well your public IP address and the port 443. I hope you get a beautiful 200 as response :)
If you don't get a successful code, go through the previous steps again and make sure you didn't misconfigure anything. From my personal experience, verify the health probe again of your app, because if your default URL returns 404, it might cause 502 error. If it's the case, either modify your probe matching condition or change the path to return a healthy response.
You can also refer to the official troubleshoot doc of Microsoft: https://docs.microsoft.com/en-us/azure/application-gateway/create-gateway-internal-load-balancer-app-service-environment
6. Set up CI/CD pipeline
Now that you have an ILB ASE living inside your VNet, it becomes impossible to use public Azure DevOps pipeline to build and deploy your websites.
Here's the official explanation: https://docs.microsoft.com/en-us/azure/app-service/environment/create-ilb-ase#publish-with-an-ilb-ase
Assuming that you already have an Azure DevOps account in your organization, let's set up the CI/CD pipeline for your protected application now.
6.1 Virtual machine
Create a virtual machine in the same VNet as your ILB ASE. Since we've already prepared the subnet for it now you just need to select the vm subnet of step 1.
6.2 DNS server
Once the VM is created, enable the DNS server feature on it.
You need to create A-records pointing to the App Service Environment's Internal Load Balancer IP address for the following hostnames (*, *.scm, ftp, publish).
Once added, set the VNet to use your VM as the DNS server (you may need to reboot your VM after the configuration)
Latest update: For ASE created after May 2019, configure the records for the *.appserviceenvironment.net domain.
6.3 Build agent
You could just use the same VM as your build agent since it's living inside the same VNet of the ILB ASE.
Simply follow this documentation for the provision: https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/v2-windows?view=azure-devops&viewFallbackFrom=vsts
6.4 Azure DevOps
Add the build agent to the Agent pools from your Azure DevOps portal.
Create a new build and release definition in VSTS pipeline by using the private build agent.
This article describes more in details: https://devopsandcloud.wordpress.com/2018/04/27/deploy-to-azure-ilb-ase-using-visual-studio-online-services/
A very important tip: if you ever encountered the certificate issue while publishing the app, you may want to set the variable VSTS_ARM_REST_IGNORE_SSL_ERRORS to true in your settings.
Summary
In this article, we covered the basics of an advanced scenario of Azure Application Gateway & ILB ASE. It helps reduce the overhead of the advanced infrastructure provision in Azure.
If you like this content, please feel free to leave a comment, share it to your network, or simply click the like button at the bottom of the article.
References
Below are the external resources that I used while working on this subject and I found them very useful as well:
https://docs.microsoft.com/en-us/azure/app-service/environment/integrate-with-application-gateway
https://sabbour.me/how-to-run-an-app-service-behind-a-waf-enabled-application-gateway/
https://medium.com/@jkudo/recommendation-of-azure-app-service-environment-f0e57c42b95a
https://docs.microsoft.com/en-us/azure/app-service/environment/network-info
Hi This blog save my day, and give me lot a help. Thank you! I have a question. When you set HTTPS for 5.5 Listner, then you must add a certificate. Which certificate to use. That for internal-mydomain.com or mydomain.com? A also have one more question: If my App (app service) custom domain is set to app-id.testing.com, a CNAME in DNS provider should be registered with app-id.testing.com point to App Gateway. A Probe should be registered point to app-id.testing.com, and a Listner hostname should be app-id.testing.com. Have I understand correct where app-id.testing.com need to be used? Thanks a lot for a great post!
Hi JF, so glad that this article helped, and you're welcome :) The current version of my website doesn't send out notifications for replies (this feature comes with the next release...I promise), hope you'll see my comment soon. Regarding your questions here is the answer: 1. When setting up the http listener, you need to upload the certificate of your public domain name, hence mydomain.com. 2. If your app has app-id.testing.com as custom domain, you should put it both in Health probe and the Listener, so yes you've understood correctly! Consider the Listener and the Health probe the public entrance to your App Gateway that will be receiving outcoming requests from outside the VNet. However, be careful with the .cer that you put in the related Http Settings, it should be the one for your internal-mydomain.com. Wish you good luck and I'm looking forward to your feedback, cheers!
Hi thanks for this great tutorial. I just have one question, in step 4 when you are creating the waf with Https, it asked me about the PFX certificate, in this point ¿have I to upload the PFX certificate of the ASE like internal-ase.com or the PFX certificate of my webapps like contoso.com? Thanks in advance!
Hi aromanrod, Glad that my post helped :) So I just checked in the portal and I see the configuration is for the http listener, so in this case, the .PFX file you need to upload is for your web app domain, like contoso.com. Good luck and looking forward to hearing from you. Cheers!