Why is a two-step synchronization required to remove a node in AKS?

NCache operates as a logical cluster within physical Kubernetes pods. The two-step method ensures the node is first logically and gracefully removed from the cluster to prevent data loss or errors, followed by physical pod removal via Kubernetes scaling to keep the physical infrastructure in sync.

Can I use Manual Scaling while Horizontal Pod Autoscaling (HPA) is active?

Yes, but you must be cautious. For maintenance removal, it is recommended to update the HPA min-replicas or the YAML manifest to ensure the scale-down is persistent and not overwritten by the HPA controller based on resource thresholds.

What happens if I scale down the Kubernetes replicas before removing the node from NCache?

This is not recommended as the cluster may view it as an ungraceful node failure. Removing the pod before the logical node is stop-cached can trigger unnecessary state transfers or lead to 'node unavailable' errors in the management tools.

Removing NCache Server(s) in Azure AKS at Runtime

NCache allows you to remove a cache server from the Azure AKS cluster at runtime by using the NCache management tools and NCache deployment. You can do so by following a two-step synchronization method: first remove the node from the NCache logical cluster via the Management Center, and then scale down the Kubernetes deployment replicas to ensure the physical infrastructure matches the cache configuration. These steps are explained in the following lines:

Important

Stop cache on the node(s) that need to be removed. You can stop cache on any node through the NCache Management Center or the NCache PowerShell Tool.

Removing cache server(s) from an NCache cluster

Cache servers can be removed from an NCache cluster using the following methods:

Follow these steps for manual scaling to remove cache servers through the NCache Management Center.
- Access the NCache Management Center from [gatewayIPAddress]:8251
- Go to Remove Server Nodes in the NCache Management Center and follow the steps mentioned there to remove cache servers from the cluster.
To remove cache servers through PowerShell, follow the steps mentioned in Remove Server Nodes through PowerShell.

Scaling Down Kubernetes Replicas in Azure AKS

After you have removed the required node(s) from the cluster, you need to decrease the replica count in the NCache deployment pod so that the changes made remain consistent with each other.

Follow these steps to decrease the replica countin the deployment file.

Manual Deployment

Execute the following command in the Azure Cloud Shell to perform Manual Scaling by decreasing the number of replicas in the deployment.

kubectl scale --replicas=number_of_remaining_server_nodes deployment/ncache

You can also manage Horizontal Pod Autoscaling configurations or manual replicas from the created YAML file ncache.yaml.
- Open ncache.yaml in a text editor and update the replicas count under the spec tag according to your requirement.
- After the YAML file has been modified, run the following command in the Azure Cloud Shell to apply the changes made to the deployment.

kubectl apply -f [dir]/ncache.yaml

Replica count can also be decreased manually by modifying ncache_custom_resource.yaml file to adjust the Manual Scaling parameters.
- Access ncache_custom_resource.yaml file by executing the following command:

kubectl edit ncache ncache

Modify clusterSize under the spec tag according to your requirement to bypass or update Horizontal Pod Autoscaling limits.

After the required cache servers have been removed from the cache cluster, you can view these changes from the NCache Management Center. You can also monitor the cluster and check its stats. See Monitor NCache Cluster and Clients.

Removing NCache Server(s) in Azure AKS at Runtime

Important

Removing cache server(s) from an NCache cluster

Scaling Down Kubernetes Replicas in Azure AKS

Manual Deployment

See Also

Contact Us