Azure Container Apps

Azure Container Apps is Azure’s solution to hosting container-based applications without the overhead of managing a Kubernetes cluster (even when the control plane is managed for you).

After using ACA for almost a year, I wanted to document my experience, as well as some of the positives and negatives that I’ve noticed.

Networking

/23 subnets are required for VNet-integrating Azure Container App Environments (EDIT - after some research, this is now only for the “legacy” Consumption-only environment, though a /23 is still recommended) - this is considerably larger than an equivalent service - App Service Plans only require a /26 (or /27 if not using a Multi-Plan subnet join). You could VNet-integrate 8 ASPs in the space that one ACAE requires!

Furthermore - a big problem I ran into was trying to implement zero trust networking - to do this properly, you need to create an ACA per ACAE. The subnet space and overhead adds up when multiple different apps are being hosted.

AzureRM Terraform Provider

I ran into a couple of issues when deploying ACAs via Terraform.

The first issue was disabling public access to my ACAE. I couldn’t even do this via the AzAPI provider - my workaround was disabling the access via an experimental az CLI extension!

The other issue I ran into was adding a User-Assigned Managed Identity to the ACAE (for facilitating access to a certificate and private key in a Key Vault). Thankfully, support for this has now been added.

Reverse Proxy / Load Balancing

A feature I do appreciate is the built-in reverse proxy and load balancing functionality. Whilst it doesn’t have the same functionality as if you were to configure a reverse proxy / ingress controller yourself, it does the basics pretty well and includes support for HTTP-based scaling. Interestingly, the scaling utilises KEDA (Kubernetes Event-Driven Autoscaling) under the hood.

Storage

ACA only supports network-level persistent storage, via Azure Files (SMB/NFS). This can cause issues with applications that require block-level storage (in my case, this was Qdrant).

I also occasionally ran into file permission issues, sometimes needing to specify specific mount permissions and/or add “nobrl” to resolve issues, for example when an application has a sqlite3 database.

Frustratingly, Azure Kubernetes Service does support block-level storage.

All of the security settings in Azure Files need disabling too :(.

ACR Weirdness

One issue I did run into was relating to using a private Azure Container Registry. As I was deploying ACAs via Terraform, utilising Managed Identities with AcrPull rights to authenticate to the ACR, I wasn’t performing the container configuration myself. As such, when I was reviewing the ACA config in the Azure Portal, I wasn’t able to see some of the container template myself, and it appeared blank / not configured.

I couldn’t see the container image name, the version, the registry’s hostname. I could still see environment variables though. This could be a permissions issue, but either way, was odd.

Summary

Overall, Azure Container Apps is a brilliant solution for teams without Kubernetes experience and/or not wanting the overhead of a K8s cluster. It could do with a bit more TLC from Microsoft but regardless, I’ve enjoyed working with it and found it to be a mostly seamless experience for deploying container-based workloads.