Introduction to the EDGE AI FOUNDATION Taxonomy

Published by the EDGE AI FOUNDATION COMMERCIALIZATION Working Group, inclUDING:

• Jason Shepherd – ATYM
• Rob Woolley – Windriver
• Eric Smiley – embedUR

The goal of the EDGE AI FOUNDATION (EDGE AI) is to build awareness and best practices for implementing artificial intelligence (AI) in edge computing use cases.

This white paper builds on the community’s “2026 and Beyond – The Edge AI Transformation”paper by establishing the EDGE AI Taxonomy to establish context for future publications. The Edge Taxonomy is driven by inherent technical and logistical tradeoffs at the various points in the edge continuum. This spans small, resource-constrained devices distributed in the physical world to large servers running in regional data centers. It then illuminates key considerations for stakeholders spanning engineering and operations with the taxonomy paradigms in mind.

Ideal Future State

The ideal future state is a common set of tools to build, deploy, and iterate AI models spanning the edge continuum. One of the objectives of the EDGE AI FOUNDATION is to raise awareness and facilitate collaboration towards this goal.  Still, there are inherent tradeoffs that this paper will discuss.

The best aspects of the cloud development experience should also apply at the edge: platform independence, code portability, continuous software delivery, tight security, and high availability when building, deploying, managing, and securing hardware and applications. For this to happen, applications and orchestration tools must be optimized for each of the inherently different edge paradigms. At the same time, they must be coordinated for overall interoperability across the continuum and address specific needs of diverse stakeholders.

This is necessary to achieve the ultimate goal of a “single pane of glass” experience where developers are free to build portable applications and deploy them securely along the cloud-to-edge continuum wherever it makes the most sense based on factors such as performance, cost, uptime, safety, and security. It is no small feat, and a key element needed to make this happen is an “orchestrator of orchestrators” model that links together all the different underlying toolsets.

When it comes to AI workloads specifically, tools and infrastructure are developed and standardized such that AI models can be built once and deployed in as many locations along the edge continuum as possible, while recognizing inherent technical and logistical tradeoffs.   As a general rule, we will continue to see the most sophisticated model training done in the cloud, but this will increasingly shift over time as more capable edge compute is deployed in production.

We will see generative AI implemented in new places and a rise of federated learning to decentralize training and help address data sovereignty requirements. In any event, the ideal state is that AI models and other applications are as portable as possible across the continuum.  In our next paper we will explore types of AI use cases that are typical across the edge continuum.

The EDGE AI Taxonomy

It’s easiest to view the Edge AI solution stack in the context of discrete edge computing paradigms. The EDGE AI community has adapted the foundation of the LF Edge taxonomy¹ which defines the edge as a continuum requiring application and infrastructure solutions to address inherent technical and logistical tradeoffs.

1. LF Edge Taxonomy (Source: Sharpening the Edge White Paper, 2020)

Key Concepts and Considerations

This section walks through general considerations for organizations building out their edge AI strategy and associated solution providers.

Application vs. Infrastructure Solution Planes

For each node in the edge continuum there are two key solution “planes” – application and infrastructure.  Software related to the acquisition, processing, and transmission of data are in the Application Plane.  Specific to edge AI, this includes applications for model training and inferencing, ML ops, data normalization, storage, and more. Meanwhile, tools for underlying Management and Orchestration (MANO) and security of both applications and devices are in the Infrastructure Plane.

Key Elements of Application and Infrastructure Solution Planes

Edge AI workloads can be applied in both planes. In the Application Plane, AI is leveraged for core data analytics that drive business outcomes and user experiences.  In the Infrastructure Plane, AI can be leveraged to analyze telemetry that reflects device health. Examples of these tools include ML models that monitor the operational metrics of the system and be used to assess system health, hardware faults, resource consumption, and security alerts.  The Infrastructure Plane may also provide the drivers and frameworks to support various AI accelerators and monitor the supported inferencing engines to help detect model drift.

Some solution providers address both planes, whereas others focus on delivering best in class solutions for specific capabilities. A key decision in your edge AI strategy is how much you abstract and partition investments across these two solution planes. While turnkey solutions may be attractive as a “one stop shop”, they may come with the tradeoff of less rich capabilities in specific areas or higher cost.  Meanwhile, discrete solutions can be integrated and optimized to best address the inherently heterogeneous nature of edge deployments.  The following sections further break down the continuum and highlight considerations for deciding which approach makes the most sense for your business and use cases.

Management and Orchestration (MANO) Paradigms

While the overall taxonomy considers factors such as solution ownership (e.g. Service Providers vs. End Users), general locality, and device type, it is also helpful to break the edge continuum down based on inherent technical and logistical tradeoffs when it comes to Management and Orchestration of both applications and hardware.  Here the EDGE AI Taxonomy also leverages the LF Edge taxonomy² while simplifying the four paradigm names.

2. LF Edge MANO Taxonomy (Source: Sharpening the Edge II: Diving Deeper into the LF Edge Taxonomy and Projects White Paper, 2022)

The four Edge MANO paradigms in the EDGE AI Taxonomy are:

1. Data Center Edge: Regional, metropolitan and on-prem data centers with a well-defined security perimeter (both physical and network) and a highly reliable connection between the orchestrator/controller and computing hardware.
2. Distributed Edge: Edge nodes deployed outside of a traditional data center environment, but still capable of supporting data center technologies like Linux, virtual machines, Docker and Kubernetes. These edge nodes are capable but will not be able to run as complex AI workloads as centralized compute. Another key difference from the Data Center Edge is that developers and admins must assume that someone can access and tamper with hardware deployed in environments such as the factory floor, a retail store, or out in an oil field. Distributed edge nodes also need to be able to run autonomously if they lose connection to their central controller (based on network availability, or by design in air-gapped scenarios).
3. End User Device Edge: Represented by UI-centric PCs, mobile devices, and wearables that have well established ecosystems like Windows, Android and iOS. These devices provide a Human-Machine Interface (HMI) to allow the End User to interact with the digital world.
4. Constrained Device Edge: Similar to the Distributed Edge, but with the added challenge of not having the system resources to run traditional data center technologies like Linux, Docker and Kubernetes. These devices are powered by microcontrollers (MCUs) and lower end MPUs and CPUs and include sensors, actuators, controllers, and cameras as well as systems such as legacy vehicles, and lightweight robots and drones. They traditionally require a real-time operating system (RTOS) linked with embedded firmware or embedded Linux; however, this is changing as new technologies such as WebAssembly-based containerization emerge.

In addition to accommodating for periodic loss of connectivity, software updates for Distributed and Constrained Device Edge assets must work in reverse of how it is typically done in the data center. While server infrastructure in the data center is typically on a trusted network with its management controller that pushes updates as needed, production devices are typically deployed on untrusted networks and behind network firewalls and proxies. Therefore, they need to be able to ping their central controller and pull updates whenever they have a connection. The Constrained Device Edge introduces additional complications due to the resource constraints of the target edge nodes. Further, the cost of network connectivity for highly distributed assets can be prohibitive when it comes to frequent software updates.

The following table highlights key differences between each MANO paradigm across a variety of factors.

Accommodating Needs for Diverse Stakeholders

The further you move down the edge continuum from the cloud, the more diverse both hardware and software become. The shift also introduces greater challenges in areas such as security (both physical and network), connectivity and safety.  While cloud computing infrastructure is relatively homogenous, hardware complexity escalates more rapidly than software as you get closer to physical environments.

Hardware variations include specific form factors, I/O and connectivity, design considerations such as ruggedization, and industry-specific certifications. This diversity begins to ramp at the Distributed Edge and grows exponentially into the Constrained Device Edge. To manage this increasing complexity, a key goal is to abstract software from hardware as much as possible so applications can remain more consistent even as hardware becomes more specialized.

You also can’t forget the people component. Specific to edge AI, there’s a difference between a data scientist, developer that trains models, one that does day to day DevOps / ML Ops and an operator in the field, and the end user.

The person that manages edge infrastructure isn’t necessarily the same person that develops and manages apps. And these tasks could be the responsibility of resources both internal and external to your organization, including third-party service providers. Or the solution could be provided by an OEM that needs to manage their products remotely but also provides access to customers for running their own apps.  The EDGE AI community recognizes that its output must address these types of logistical considerations spanning the edge continuum.

Historically there has been a physical separation of concerns between IT infrastructure such as an edge server or gateway and OT infrastructure including machines, controllers, actuators, and sensors. OT infrastructure has served a dedicated purpose, with IT infrastructure above abstracting data to perform alerting and analytics. Reason being, uptime and safety are key needs for OT.

Taxonomy Alignment with the Purdue Model

However, we’re increasingly seeing the lines between OT and IT blurring. An Edge AI model may be deployed on a software-defined PLC in addition to control logic. Or a sensor that is attached to a piece of equipment “out of band” from the core control system, for example to perform analytics on temperature and vibration data for a predictive maintenance use case. This a simpler starting point because there isn’t risk to machine uptime when addressing application and device management for the sensor.

These challenges translate to all verticals, whether it be smart buildings, transportation and logistics, retail, agriculture and consumer.  Included in this are considerations when dealing with the boundaries between public and private concerns. In the case of smart cities, you have to balance public infrastructure with the personal needs of citizens in terms of privacy. When collaborating with other partners in an ecosystem, you also have to work through challenges that may arise based on different business needs.

Cybersecurity

Edge computing is about connecting formerly isolated devices and systems to broader networks for business gain through visibility and analytics. However, the value realized must be greater than the risk involved. As such, cybersecurity is a top priority for the EDGE AI community as it develops a framework for interoperable industrial edge computing orchestration.

Fundamental to cybersecurity at the edge is the concept of zero trust, meaning all hardware and software is untrusted by default and access and connections between these resources are only made on an exception-level basis. Adopting a zero-trust security model is especially key for deployments at the Distributed and Constrained Device Edges because there is often no defined physical or network perimeter, as is typical at the Data Center Edge.

Technical limitations within different edge paradigms also need to be considered. For example, resource-constrained devices may not have a secure element available to serve as Root of Trust, and if they do, it is likely not a Trusted Platform Module (TPM) that is common on more capable devices that can run a full operating system like Linux or Windows.

The concerns among OT and IT professionals also vary. While OT typically cares most about uptime, quality, and safety, IT is often most concerned about protecting data. The ramifications of a security breach also tend to play out in very different ways. OT attacks tend to have immediate impact on production and potentially safety, whereas IT security breaches play out over long periods of time (for example, in a credit card data leak).

While a brief downtime caused by an IT update on a server or PC may be a minor annoyance, coordinating updates for critical infrastructure is essential to ensure both production uptime and safety.

When security measures are overly burdensome, users are tempted to figure out workarounds that can introduce vulnerabilities and overall adoption can be hindered.  For that reason, the EDGE AI places strong emphasis on security usability. Areas of focus include accommodating a mix of skill sets, enabling zero touch provisioning, and reducing manual entry of configurations.

The EDGE AI community is also focused on ensuring that deployments will integrate seamlessly with existing enterprise security infrastructure. This means supporting standard enterprise firewalls, working through proxy servers that perform inspection/content filtering and integrating with existing Public Key Infrastructure (PKI) for device identity, secure communications, and signing. The goal is to maintain security standards while working within established enterprise frameworks.

Finally, the EDGE AI community is looking to ease the burden for organizations in meeting key regulatory requirements such as Software Bill of Materials (SBOM), the Cyber Resilience Act (CRA), and emerging guidelines around the use of memory-safe programming languages.

Conclusion

“The Edge” isn’t one thing: rather, it’s a continuum spanning regional data centers to resource constrained devices and systems in the physical world such as sensors, smart cameras, appliances, drones, robots, vehicles, and beyond. Use cases for Edge AI and the associated technical implementations vary based on inherent technical and logistical trade-offs across this continuum.

Over the past 10 years, we’ve seen data center technologies and development principles being extended from the cloud to various edge locations. However, there’s a practical limit to as far down the continuum that these technologies are applicable. In a perfect world, we would be able to have a singular tool set that helps us build AI models and deploy them anywhere along the edge continuum, based on a balance of performance, cost, security, and privacy. One pane of glass to manage models and devices across the continuum.

The practical reality is that getting to a unified edge AI experience requires a “system of systems” approach that unifies inherently different tools spanning applications and AI to underlying infrastructure that are required for each paradigm in the continuum. In terms of AI, we need tools that share a common workflow but also adapt to the specific capabilities of the target devices. Specific to infrastructure, while technologies like virtual machines, Docker, and Kubernetes are well-suited for more compute-capable edge nodes, they don’t extend down to the billions of resource-constrained embedded devices.  Emerging technologies, such as WebAssembly, are enabling the same principles of containerization and CI/CD to be extended to highly resource-constrained devices, including microcontrollers.

The purpose of this taxonomy is to create a foundation to frame conversations about edge AI use cases, enabling technologies, and roles required to make edge AI a reality.  The taxonomy focuses on absolutes instead of vague terms like “near and far” edges and “thin and thick” edges. It breaks the continuum down into a variety of vectors such as whether the edge location is owned or operated by a service provider or an end user, where specifically along the continuum the edge node is deployed, the different considerations spanning the application and infrastructure planes of a solution, and inherent tradeoffs between different management and orchestration paradigms spanning edge data centers to distributed edge deployments, end user devices, and highly constrained devices.

A future paper will outline a solution matrix of different providers within the ecosystem and map them to this taxonomy. This includes providers that specialize in AI specifically, as well as critical elements such as security, management and orchestration, hardware and silicon, and services that make edge AI possible in the physical world at scale.