The four key considerations for deploying applications to the edge are connectivity, architecture, sizing and scale, according to Mikhail Kozorovitskiy, consulting engineer at Rancher Labs.
Kozorovitskiy explained these considerations as well as how to solve common issues they run into at the Computing on the Edge with Kubernetes conference yesterday.
According to Kozorovitskiy, the four considerations can be broken down further into the following choices:
- Connectivity: Air gapped, semi-connected, fully connected
- Architecture: Arm, x86
- Sizing: Small, medium, and large
- Scale: Mostly manual, mostly automated
During his talk, Kozorovitskiy provided examples of the common issues for each, along with how to solve those issues. For instance, issues that could arise in air-gapped deployments include data not transmitting out, difficulties in updating and deploying, and difficulties in testing. For semi-connected workloads, challenges are that there is a centrally unknown state, they are hard to update and deploy, and speed could be inconsistent. For fully connected devices, security is the main issue.
To solve issues around deploying and updating in an air-gapped environment, Kozorovitskiy recommended preloading or locally deploying applications. “To preload locally utilize an image registry and chart catalog or just load the images directly into the container runtime to minimize overhead,” he said. “Updating local requires either a controlled local web interface or shell level access. A privileged microservice can load new images and applications into your edge device.”
To overcome testing issues in air-gapped environments, he recommended utilizing network rules to create an air-gapped-like environment.
To solve issues in semi-connected workloads Kozorovitskiy recommended using a pull type state management to validate the state of an edge device. This can be done using tools like Rancher 2.5 using the Continuous Delivery function or Fleet. He also recommends doing batch transmission jobs to solve for inconsistencies in speed.
To address security issues in fully connected devices there are a number of solutions. These include employing firewalls and network policies to limit access between unrelated workloads, blocking inbound access unless it is required, ensuring data is encrypted in transit, using restrictive pod security policies, using OPA to validate security requirements, scanning images for vulnerabilities, and utilizing Kubernetes security best practices.
Moving onto architecture, Kozorovitskiy explained that a common issue for x86 architectures is power consumption, and issues for Arm are that it is hard to develop on x86 and deploy on Arm, and there are limited third party images.
Kozorovitskiy recommended still using x86 even despite the fact that it is less efficient, unless power consumption is an important factor. To develop on x86 and deploy on Arm, a few extra steps are needed, such as using multi-arch build pipelines. And to overcome the issue of limited third party images on Arm, he recommended building your own versions of those apps or checking that the tools you want to use have arm versions.
Different issues arise based on size as well. For example, small devices have resource limitations, medium devices can have legacy workloads, and large devices can have a large node configuration, multi-node configuration, and dynamic provisioning.
To overcome resource limitations on small devices, Kozorovitskiy recommended sticking to minimalist workloads and utilizing a minimalist OS and Kubernetes installations. For medium-sized devices, use VMs to handle legacy workloads and use the strangler pattern to isolate them from utilizing standard probes. Kubevirt is a tool that can help with this issue. For large-sized devices, he recommended modifying standard Kubernetes configurations for large node deployments to allow for more than 110 pods per node, which is the default maximum.
The final consideration is scale. Kozorovitskiy cautions against manual deployments because there are several issues. Issues in automated deployments include deploying without upgradability for the core system, deploying without the ability to update workloads, and observability.
He believes that whether you have one or one million edge deployments, the key to success is knowing and testing upgrades. According to Kozorovitskiy, air-gapped installations need to be updated on location, whereas semi-connected edge installations are more flexible and can allow for edge triggered configuration updates.
