As the first cloud-native edge computing community, KubeEdge provides solutions for cloud-edge synergy and has been widely adopted in industries including Transportation, Energy, Internet, CDN, Manufacturing, Smart campus, etc. With the accelerated deployment of KubeEdge in this area based on cloud-edge synergy, the community will improve the security of KubeEdge continuously in cloud-native edge computing scenarios.

The KubeEdge community attaches great importance to security and has set up Sig Security and Security Team to design KubeEdge system security and quickly respond to and handle security vulnerabilities. To conduct a more comprehensive security assessment of the KubeEdge project, the KubeEdge community cooperates with Ada Logics Ltd. and The Open Source Technology Improvement Fund performed a holistic security audit of KubeEdge and output a security auditing report, including the security threat model and security issues related to the KubeEdge project. Thank you to experts Adam Korczynski and David Korczynski of Ada Logics for their professional and comprehensive evaluation of the KubeEdge project, which has important guiding significance for the security protection of the KubeEdge project. Thank you Amir Montazery and Derek Zimmer of OSTIF and Cloud Native Computing Foundation (CNCF) who helped with this engagement.

The discovered security issues have been fixed and patched to the latest three minor release versions (v1.11.1, v1.10.2, v1.9.4) by KubeEdge maintainers according to the kubeedge security policy. Security advisories have been published here.

For more details of the threat model and the mitigations, Please check KubeEdge Threat Model And Security Protection Analysis: https://github.com/kubeedge/community/tree/master/sig-security/sig-security-audit/KubeEdge-threat-model-and-security-protection-analysis.md.

References:​

Audit report: https://github.com/kubeedge/community/tree/master/sig-security/sig-security-audit/KubeEdge-security-audit-2022.pdf

OSTIF Blogpost: https://ostif.org/our-audit-of-kubeedge-is-complete-multiple-security-issues-found-and-fixed

CNCF Blogpost:

KubeEdge Threat Model And Security Protection Analysis: https://github.com/kubeedge/community/tree/master/sig-security/sig-security-audit/KubeEdge-threat-model-and-security-protection-analysis.md

On Jun 21, 2022 KubeEdge released v1.11, introducing several exciting new features and enhancements that significantly improve node group management, mapper development, installation experience, and overall stability.

v1.11 What's New​

• Node Group Management
• Mapper SDK
• Beta sub-commands in Keadm to GA
• Deprecation of original init and join
• Next-gen Edged to Beta: Suitable for more scenarios

Release Highlights​

Node Group Management​

Users can now deploy applications to several node groups without writing deployment for every group. Node group management helps users to:

• Manage nodes in groups

• Spread apps among node groups

• Run different versions of app instances in different node groups

• Limit service endpoints in the same location as the client

Two new APIs have been introduced to implement Node Group Management:

• NodeGroup API: represents a group of nodes that have the same labels.
• EdgeApplication API: contains the template of the application organized by node groups, and the information on how to deploy different editions of the application to different node groups.

Refer to the links for more details (#3574, #3719).

Mapper SDK​

Mapper-sdk is a basic framework written in Go. Based on this framework, developers can more easily implement a new mapper. Mapper-sdk has realized the connection to KubeEdge, provides data conversion, and manages the basic properties and status of devices, etc., as well as basic capabilities and abstract definition of the driver interface. Developers only need to implement the customized protocol driver interface of the corresponding device to realize the function of mapper.

Refer to the link for more details (#70).

Beta sub-commands in Keadm to GA​

Some new sub-commands in Keadm have moved to GA, including containerized deployment, offline installation, etc. The original init and join behaviors have been replaced by the implementation from beta init and beta join:

• CloudCore will be running in containers and managed by Kubernetes Deployment by default.

• Keadm now downloads releases that are packed as container images to edge nodes for node setup.

• init: CloudCore Helm Chart is integrated into init, which can be used to deploy containerized CloudCore.

• join: Installing edgecore as a system service from a Docker image, no need to download from the GitHub release.

• reset: Reset the node, clean up the resources installed on the node by init or join. It will automatically detect the type of node to clean up.

• manifest generate: Generate all the manifests to deploy the cloud-side components.

Refer to the link for more details (#3900).

Deprecation of original init and join​

The original init and join sub-commands have been deprecated as they had issues with offline installation, etc.

Refer to the link for more details (#3900).

Next-gen Edged to Beta: Suitable for more scenarios​

The new version of the lightweight engine Edged, optimized from Kubelet and integrated into edgecore, has moved to Beta. The new Edged will still communicate with the cloud through the reliable transmission tunnel.

Refer to the link for more details (Dev-Branch for beta: feature-new-edged).

Important Steps before Upgrading​

If you want to use Keadm to deploy KubeEdge v1.11.0, please note that the behaviors of the init and join sub-commands have been changed.

Other Notable Changes​

• Add custom image repo for keadm join beta (#3654)

• Keadm: beta join support remote runtime (#3655)

• Use sync mode to update pod status (#3658)

• Make log level configurable for local up kubeedge (#3664)

• Use dependency to pull images (#3671)

• Move apis and client under kubeedge/cloud/pkg/ to kubeedge/pkg/ (#3683)

• Add subresource field in application for API with subresource (#3693)

• Add Keadm beta e2e (#3699)

• Keadm beta config images: support remote runtime (#3700)

• Use unified image management (#3720)

• Use armhf as default for armv7/v6 (#3723)

• Add ErrStatus in api-server application (#3742)

• Support compile binaries with kubeedge/build-tools image (#3756)

• Add min TLS version for stream server (#3764)

• Adding security policy (#3778)

• Chart: add cert domain config in helm chart (#3802)

• Add domain support for certgen.sh (#3808)

• Remove default KubeConfig for cloudcore (#3836)

• Helm: Allow annotation of the cloudcore service (#3856)

• Add rate limiter for edgehub (#3862)

• Sync pod status immediately when status update (#3891)

On Mar 7, 2022, KubeEdge released v1.10. The new version introduces several enhancements, significantly improving the installation experience, performance testing, network communication, and Kubernetes version compatibility.

v1.10 What's New​

• Installation Experience Improvement with Keadm
• Preview version for Next-gen Edged: Suitable for more scenarios
• Edgemark: Support large-scale KubeEdge cluster performance testing
• EdgeMesh proxy tunnel supports quic
• EdgeMesh supports proxy for udp applications
• EdgeMesh support SSH login between cloud-edge/edge-edge nodes
• Kubernetes Dependencies Upgrade

Release Highlights​

Installation Experience Improvement with Keadm​

Keadm adds some new sub-commands to improve the user experience, including containerized deployment, offline installation, etc. New sub-commands including: beta, config.

beta provides some sub-commands that are still in testing, but have complete functions and can be used in advance. Sub-commands including: beta init, beta manifest generate, beta join, beta reset.

• beta init: CloudCore Helm Chart is integrated in beta init, which can be used to deploy containerized CloudCore.

• beta join: Installing edgecore as system service from docker image, no need to download from github release.

• beta reset: Reset the node, clean up the resources installed on the node by beta init or beta join. It will automatically detect the type of node to clean up.

• beta manifest generate: Generate all the manifests to deploy the cloudside components.

config is used to configure kubeedge cluster, like cluster upgrade, API conversion, image preloading. Now the image preloading has supported, sub-commands including: config images list, config images pull.

• config images list: List all images required for kubeedge installation.

• config images pull: Pull all images required for kubeedge installation.

Refer to the links for more details. (#3517, #3540, #3554, #3534)

Preview version for Next-gen Edged: Suitable for more scenarios​

A new version of the lightweight engine Edged, which is optimized from kubelet and integrated in edgecore, and occupies less resource. Users can customize lightweight optimization according to their needs.

Refer to the links for more details. (Dev-Branch for previewing: feature-new-edged)

Edgemark: Support large-scale KubeEdge cluster performance testing​

Edgemark is a performance testing tool inherited from Kubemark. The primary use case of Edgemark is also scalability testing, it allows users to simulate edge clusters, which can be much bigger than the real ones.

Edgemark consists of two parts: real cloud part components and a set of "Hollow" Edge Nodes. In "Hollow" Edge Nodes, EdgeCore runs in container. The edged module runs with an injected mock CRI part that doesn't do anything. So the hollow edge node doesn't actually start any containers, and also doesn't mount any volumes.

Refer to the link for more details. (#3637)

EdgeMesh proxy tunnel supports quic​

Users can choose edgemesh's proxy tunnel as quic protocol to transmit data. In edge scenarios, nodes are often in a weak network environment. Compared with the traditional tcp protocol, the quic protocol has better performance and QoS in the weak network environment.

Refer to the link for more details. (#281)

EdgeMesh supports proxy for udp applications​

Some users' services use the udp protocol, and now edgemesh can also support the proxy of udp applications.

Refer to the link for more details. (#295)

EdgeMesh support SSH login between cloud-edge/edge-edge nodes​

Edge nodes are generally distributed in the Private network environment, but it is often necessary to ssh login and operate the edge node. EdgeMesh provide a socks5proxy based on the tunnel inside EdgeMesh, which supports forwarding ssh requests from cloud/edge nodes to edge nodes.

Refer to the links for more details. (#258, #242)

Kubernetes Dependencies Upgrade​

Upgrade the vendered kubernetes version to v1.22.6, users now can use the feature of new version on the cloud and on the edge side.

Refer to the link for more details. (#3624)

Important Steps before Upgrading​

If you want to deploy the KubeEdge v1.10.0, please note that the Kubernetes dependency is 1.22.6.

Other Notable Changes​

• Remove dependency on os/exec and curl in favor of net/http (#3409, @mjlshen)

• Optimize script when create stream cert (#3412, @gujun4990)

• Cloudhub: prevent dropping volume messages (#3457, @moolen)

• Modify the log view command after edgecore is running (#3456, @zc2638)

• Optimize the iptables manager (#3461, @zhu733756)

• Add script for build release (#3467, @gy95)

• Using lateset codes to do keadm_e2e (#3469, @gy95)

• Change the resourceType of msg issued by synccontroller (#3496, @Rachel-Shao)

• Add a basic image for building various components of KubeEdge (#3513, @zc2638)

• Supporting crossbuild all components (#3515, [@fisher

KubeEdge is an open source system extending native containerized application orchestration and device management to hosts at the Edge. It is built upon Kubernetes and provides core infrastructure support for networking, application deployment and metadata synchronization between cloud and edge. It also supports MQTT and allows developers to author custom logic and enable resource constrained device communication at the Edge.

On December 6th, the KubeEdge community is proud to announce the availability of KubeEdge 1.9. This release includes a major upgrade for Custom HTTP Request Routing from Edge to Cloud through ServiceBus for Applications, CloudCore run independently of the Kubernetes Master host and containerized deployment using Helm, EdgeMesh add tls and encryption security, and compiled into rpm package, which includes:

• Custom HTTP Request Routing from Edge to Cloud through ServiceBus for Applications

• CloudCore run independently of the Kubernetes Master host

• EdgeMesh add tls and encryption security

• Enhance the ease of use of EdgeMesh

• Support containerized deployment of CloudCore using Helm

• Support compiled into rpm package and installed on OS such as openEuler using yum package manager

• 40+ bug fixes and enhancements.

Please refer to CHANGELOG v1.9 for a full list of features in this release

Release Highlights​

Support Custom HTTP Request Routing from Edge to Cloud through ServiceBus for Applications​

A HTTP server is added to ServiceBus, to support custom http request routing from edge to cloud for applications. This simplifies the rest api access with http server on the cloud while client is in the edge.

Refer to the links for more details. (#3254, #3301)

Support CloudCore to run independently of the Kubernetes Master host​

CloudCore now supports to run independently of the Kubernetes Master host, iptablesmanager has been added as an independent component, users only need to deploy the iptablesmanager to Kubernetes Master host, which now can add the iptable rules for Cloud-Edge tunnel automatically

Refer to the links for more details. (#3265)

EdgeMesh add tls and encryption security​

EdgeMesh's tunnel module adds tls and encryption security capabilities. These features bring more secure protection measures to the user's edgemesh-server component and reduce the risk of edgemesh-server being attacked.

Refer to the links for more details. (EdgeMesh#127)

Enhanced the ease of use of EdgeMesh​

EdgeMesh has many improvements in ease of use. Now users can easily deploy EdgeMesh's server and agent components with a single command of helm. At the same time, the restriction on service port naming is removed, and the docker0 dependency is removed, making it easier for users to use EdgeMesh.

Refer to the links for more details. (EdgeMesh#123, EdgeMesh#126, EdgeMesh#136, EdgeMesh#175)

Support containerized deployment of CloudCore using Helm​

CloudCore now supports containerized deployment using Helm, which provides better containerized deployment experience.

Refer to the links for more details. (#3265)

Support compiled into rpm package and installed on OS such as openEuler using yum package manager​

KubeEdge now supports compiled into rpm package and installed on OS such as openEuler using yum package manager.

Refer to the links for more details. (#3089, #3171)

In addition to the above new features, KubeEdge v1.9 also includes the following enhancements:

• Rpminstaller: add support for openEuler (#3089)

• Replaced 'kubeedge/pause' with multi arch image (#3114)

• Make meta server addr configurable (#3119)

• Added iptables to Dockerfile and made cloudcore privileged (#3129)

• Added CustomInterfaceEnabled and CustomInterfaceName for edgecore (#3130)

• Add experimental feature (#3131)

• Feat(edge): node ephemeral storage info (#3157)

• Support envFrom configmap in edge pods (#3176)

• Update golang to 1.16 (#3190)

• Metaserver: support shutdown server graceful (#3239)

• Support labelselector for metaserver (#3262)

Future Outlook​

With the release of v1.9, KubeEdge supports custom HTTP request routing from Edge to Cloud through ServiceBus for applications, supports CloudCore running independently of the Kubernetes Master host, supports containerized deployment of CloudCore using Helm, supports tls and encryption security and the ease of use of EdgeMesh. Thanks to Huawei, China Unicom, DaoCloud, Zhejiang University SEL Lab, ARM and other organizations for their contributions, as well as all community contributors for their support!

The community plans to further improve the user experience and the stability of KubeEdge in subsequent versions and create the best “open source” intelligent edge computing platform for everyone to freely use.

For more details regarding KubeEdge, please follow and join us here:

https://kubeedge.io

KubeEdge is an open source system extending native containerized application orchestration and device management to hosts at the Edge. It is built upon Kubernetes and provides core infrastructure support for networking, application deployment and metadata synchronization between cloud and edge. It also supports MQTT and allows developers to author custom logic and enable resource constrained device communication at the Edge.

On October 31st, the KubeEdge community is proud to announce the availability of KubeEdge 1.8. This release includes a major upgrade for Active-Active HA Support of CloudCore for Large Scale Cluster, EdgeMesh Architecture Modification, EdgeMesh Cross LAN Communication, and Kubernetes Dependencies Upgrade, which includes:

• Active-Active HA Support of CloudCore for Large Scale Cluster [Beta]

• EdgeMesh Architecture Modification

• EdgeMesh Cross LAN Communication

• Onvif Device Mapper

• Kubernetes Dependencies Upgrade

• 30+ bug fixes and enhancements.

Please refer to CHANGELOG v1.8 for a full list of features in this release

Release Highlights​

Active-Active HA Support of CloudCore for Large Scale Cluster [Beta]​

CloudCore now supports Active-Active HA mode deployment, which provides better scalability support for large scale clusters. Cloud-Edge tunnel can also work with multiple CloudCore instances. CloudCore now can add the iptable rules for Cloud-Edge tunnel automatically.

Refer to the links for more details. (#1560, #2999)

EdgeMesh Architecture Modification​

EdgeMesh now has two parts: edgemesh-server and edgemesh-agent. The edgemesh-server requires a public IP address, when users use cross lan communication, it can act as a relay server in the LibP2P mode or assist the agent to establish p2p hole punching. The edgemesh-agent is used to proxy all application traffic of user nodes, acts as an agent for communication between pods at different locations.

Refer to the links for more details. (edgemesh#19)

EdgeMesh Cross LAN Communication​

Users can use cross LAN communication feature to implement cross LAN edge to edge application communication and cross LAN edge to cloud application communication.

Refer to the links for more details. (edgemesh#26, edgemesh#37, edgemesh#57)

Onvif Device Mapper​

Onvif Device Mapper with Golang implementation is provided, based on new Device Mapper Standard. Users now can use onvif device mapper to manage the ONVIF IP camera.

Refer to the links for more details. (mappers-go#48)

Kubernetes Dependencies Upgrade​

Upgrade the vendered kubernetes version to v1.21.4, users now can use the feature of new version on the cloud and on the edge side.

Refer to the links for more details. (#3021, #3034)

In addition to the above new features, KubeEdge v1.8 also includes the following enhancements:

• Refactor edgesite: import functions and structs instead of copying code (#2893)

• Avoiding update cm after created a new cm (#2913)

• Solved the checksum file download problem when ke was installed offline (#2909)

• cloudcore support configmap dynamic update when the env of container inject from configmap or secret (#2931)

• Remove edgemesh from edgecore (#2916)

• keadm: support customsized labels when use join command (#2827)

• support k8s v1.21.X (#3021)

• Handling node/*/membership/detail (#3025)

• sync the response message unconditionally (#3014)

• support default NVIDIA SMI command (#2680)

Future Outlook​

With the release of v1.8, KubeEdge supports Active-Active HA mode deployment, which provides better scalability support for large scale clusters, supports cross LAN communication by EdgeMesh, and supports Onvif Device Mapper. Thanks to Huawei, China Unicom, DaoCloud, Zhejiang University SEL Lab, ARM and other organizations for their contributions, as well as all community contributors for their support!

The community plans to further improve the user experience and the stability of KubeEdge in subsequent versions and create the best “open source” intelligent edge computing platform for everyone to freely use.

For more details regarding KubeEdge, please follow and join us here:

https://kubeedge.io