IEEE International Conference on Cloud Engineering

Virtualization of operating systems provides a common way to run different services in the cloud. Recently, the lightweight virtualization technologies claim to offer superior performance. In this paper, we present a detailed performance comparison of traditional hypervisor based virtualization and new lightweight solutions. In our measurements, we use several benchmarks tools in order to understand the strengths, weaknesses, and anomalies introduced by these different platforms in terms of processing, storage, memory and network. Our results show that containers achieve generally better performance when compared with traditional virtual machines and other recent solutions. Albeit containers offer clearly more dense deployment of virtual machines, the performance difference with other technologies is in many cases relatively small.

PaaS vendors face challenges in efficiently providing services with the growth of their offerings. In this paper, we explore how PaaS vendors are using containers as a means of hosting Apps. The paper starts with a discussion of PaaS Use case and the current adoption of Container based PaaS architectures with the existing vendors. We explore various container implementations - Linux Containers, Docker, Warden Container, lmctfy and OpenVZ. We look at how each of this implementation handle Process, FileSystem and Namespace isolation. We look at some of the unique features of each container and how some of them reuse base Linux Container implementation or differ from it. We also explore how IaaSlayer itself has started providing support for container lifecycle management along with Virtual Machines. In the end, we look at factors affecting container implementation choices and some of the features missing from the existing implementations for the next generation PaaS.

Serverless computing platforms provide function(s)-as-a-Service (FaaS) to end users while promising reduced hosting costs, high availability, fault tolerance, and dynamic elasticity for hosting individual functions known as microservices. Serverless Computing environments, unlike Infrastructure-as-a-Service (IaaS) cloud platforms, abstract infrastructure management including creation of virtual machines (VMs), operating system containers, and request load balancing from users. To conserve cloud server capacity and energy, cloud providers allow hosting infrastructure to go COLD, deprovisioning containers when service demand is low freeing infrastructure to be harnessed by others. In this paper, we present results from our comprehensive investigation into the factors which influence microservice performance afforded by serverless computing. We examine hosting implications related to infrastructure elasticity, load balancing, provisioning variation, infrastructure retention, and memory reservation size. We identify four states of serverless infrastructure including: provider cold, VM cold, container cold, and warm and demonstrate how microservice performance varies up to 15x based on these states.

Emerging container technologies, such as Docker, offer unprecedented agility in developing and running applications in cloud environment especially when combined with a microservice-style architecture. However, it is often difficult to use containers to manage the cloud infrastructure, without sacrificing many benefits container offers. This paper identifies the key challenges that impede realizing the full promise of containerizing infrastructure services. Using OpenStack as a case study, we explore solutions to these challenges. Specifically, we redesign OpenStack deployment architecture to enable dynamic service registration and discovery, explore different ways to manage service state in containers, and enable containers to access the host kernel and devices. We quantify the efficiency of the container-based microservice-style DevOps compared to the VM-based approach, and study the scalability of the stateless and stateful containerized components. We also discuss limitations in our current design, and highlight open research problems that, if solved, can lead to wider adoption of containers in cloud infrastructure management.

Serverless computing has emerged as a compelling paradigm for the development and deployment of a wide range of event based cloud applications. At the same time, cloud providers and enterprise companies are heavily adopting machine learning and Artificial Intelligence to either differentiate themselves, or provide their customers with value added services. In this work we evaluate the suitability of a serverless computing environment for the inferencing of large neural network models. Our experimental evaluations are executed on the AWS Lambda environment using the MxNet deep learning framework. Our experimental results show that while the inferencing latency can be within an acceptable range, longer delays due to cold starts can skew the latency distribution and hence risk violating more stringent SLAs.

EdgeAI (Edge computing based Artificial Intelligence) has been most actively researched for the last few years to handle variety of massively distributed AI applications to meet up the strict latency requirements. Meanwhile, many companies have released edge devices with smaller form factors (low power consumption and limited resources) like the popular Raspberry Pi and Nvidia's Jetson Nano for acting as compute nodes at the edge computing environments. Although the edge devices are limited in terms of computing power and hardware resources, they are powered by accelerators to enhance their performance behavior. Therefore, it is interesting to see how AI-based Deep Neural Networks perform on such devices with limited resources. In this work, we present and compare the performance in terms of inference time and power consumption of the four SoCs: Asus Tinker Edge R, Raspberry Pi 4, Google Coral Dev Board, Nvidia Jetson Nano, and one microcontroller: Arduino Nano 33 BLE, on different deep learning models and frameworks. We also provide a method for measuring power consumption, inference time and accuracy for the devices, which can be easily extended to other devices. Our results showcase that, for Tensorflow based quantized model, the Google Coral Dev Board delivers the best performance, both for inference time and power consumption. For a low fraction of inference computation time, i.e. less than 29.3% of the time for MobileNetV2, the Jetson Nano performs faster than the other devices.

Improving resource provisioning of heterogeneous cloud infrastructures is an important research challenge. The wide diversity of cloud-based applications and customers with different QoS requirements have recently exhibited the weaknesses of current provisioning systems. Today's cloud infrastructures provide provisioning systems that dynamically adapt the computational power of applications by adding or releasing resources. Unfortunately, these scaling systems are fairly limited:(i) They restrict themselves to a single type of resource, (ii)they are unable to fulfill QoS requirements in face of spiky workload, and (iii) they offer the same QoS level to all their customers, independent of customer preferences such as different levels of service availability and performance. In this paper, we present an autoscaling system that overcomes these limitations by exploiting heterogeneous types of resources, and by defining multiple levels of QoS requirements. The proposed system selects a resource scaling plan according to both workload and customer requirements. Our experiments conducted on both public and private infrastructures show significant reductions in QoS-level violations when faced with highly variable workloads.

This paper proposes a cloud-based framework and platform for end-to-end development and lifecycle management of artificial intelligence (AI) applications. We build on our previous work on platform-level support for cloud-managed deep learning services, and show how the principles of software lifecycle management can be leveraged and extended to enable automation, trust, reliability, traceability, quality control, and reproducibility of AI pipelines. Based on a discussion of use cases and current challenges, we describe a framework for managingAI application lifecycles and its key components. We also show concrete examples that illustrate how this framework enables managing and executing model training and continuous learning pipelines while infusing trusted AI principles.

The convergence of mobile computing and cloud computing enables new multimedia applications that are both resource-intensive and interaction-intensive. For these applications, end-to-end network bandwidth and latency matter greatly when cloud resources are used to augment the computational power and battery life of a mobile device. We first present quantitative evidence that this crucial design consideration to meet interactive performance criteria limits data center consolidation. We then describe an architectural solution that is a seamless extension of today's cloud computing infrastructure.

Cloud offerings are increasingly serving workloads with a large variability in terms of compute, storage and networking resources. Computing requirements (all the way to High Performance Computing or HPC), criticality, communication intensity, memory requirements, and scale can vary widely. Virtual Machine (VM) placement and consolidation for effective utilization of a common pool of resources for efficient execution of such diverse class of applications in the cloud is challenging, resulting in higher cost and missed Service Level Agreements (SLAs). For HPC, current cloud providers either offer dedicated cloud with dedicated nodes, losing out on consolidation benefits of virtualization, or use HPC-agnostic cloud scheduling resulting in poor HPC performance. In this work, we address application-aware allocation of n VM instances (comprising a single job request) to physical hosts from a single pool. We design and implement an HPC-aware scheduler on top of Open Stack Compute (Nova) and also incorporate it in a simulator (Cloud Sim). Through various optimizations, specifically topology- and hardware-awareness, cross-VM interference accounting and application-aware consolidation, we demonstrate enhanced VM placements which achieve up to 45% improvement in HPC performance and/or 32% increase in job throughput while limiting the effect of jitter (or noise) to 8%.

Burstable instances offer a cost-efficient alternative to standard cloud instances by enabling short bursts of high CPU performance via a credit-based system. However, their limited ability to sustain long-term computational loads introduces challenges for continuous processing tasks. Stream Processing Systems (SPS), which handle real-time, unbounded data streams, often originating from highly dynamic and bursty sources, require flexible and responsive resource allocation. Given their workload variability, SPS applications present a promising opportunity for cost optimization through the strategic use of burstable instances. In this work, we examine the capabilities and limitations of deploying SPS tasks on burstable instances, assessing their effectiveness in maintaining performance while reducing costs. Our findings demonstrate that the limited resources of burstable instances can significantly impact processing latency. However, when correctly balanced in instance deployment and load distribution, they can achieve notable cost savings with low impact on service quality.

We demonstrate a new customizable toolchain for validating Kubernetes manifests. The toolchain supports multiple types of validation, including validations against JSON schema, Kubernetes built-in rules, best practices, and custom business rules expressed in Common Expression Language (CEL). It also enables the detection of inconsistencies within a defined set of rules. The main goal is to highlight the value of each validation type and demonstrate how their combination improves the robustness of Kubernetes manifests, ensuring reliable deployments.

CPILS is a sustainable education system consisting of a private cloud core and peripherals for the interaction with users and the environment. It is aimed at upskilling students on societally relevant topics as well as technology to overcome uncertainties and achieve autarky. Currently, two instances of CPILS are deployed in universities as immersive learning spaces for mixed-discipline students. In this paper, we present the adaptive software design and technology decisions behind CPILS and demonstrate the resulting implementation.

Increasing demands for low latency, cost efficiency, and digital sovereignty are driving organisations beyond centralised cloud models towards decentralised and hybrid computing environments. The Edge-to-Cloud computing continuum aims to unify edge and cloud infrastructures into a seamless execution environment, but achieving this remains complex in practice.ColonyOS is an open-source, ready-to-use meta-operating system that coordinates distributed workloads across diverse computing environments. ColonyOS separates coordination from execution using declarative, intent-based function specifications and a lightweight, broker-based architecture that provides OS-like abstractions. This paper outlines key properties of a meta-operating system and describes how these are realised in ColonyOS. The paper also presents practical experiences from two industrial deployments: real-time seismic monitoring at RockSigma AB and integration with EuroHPC supercomputers. These use cases demonstrate ColonyOS’s capabilities in enabling resilient workload management and hybrid orchestration across HPC and Kubernetes environments. A referenced study on carbon-aware scheduling further illustrates how ColonyOS supports time-shifting of non-urgent workloads to reduce environmental impact. The paper concludes with a discussion of ColonyOS’s architecture and future research directions.

Serverless computing simplifies application deployment by removing the need for infrastructure management, with REST APIs being the common interface. Data persistence is essential for serverless applications because serverless functions are stateless and short-lived. To retain information across function executions, relational databases are commonly used to persist data, to ensure continuity, scalability, and reliable management. However, REST can lead to inefficiencies such as data over-fetching and under-fetching, which impact performance. GraphQL is a data query and manipulation language that supports client queries that specify what data is to be retrieved or modified. GraphQL resolvers fetch and transform data as specified by client queries.This paper compares the performance and scalability of GraphQL APIs as a database interface middleware in contrast to REST APIs implemented using serverless functions. We compare a GraphQL API implemented using the managed GraphQL AWS AppSync service, an unmanaged GraphQL API hosted using Apollo Server, and a traditional REST API implemented via the Amazon API Gateway and AWS Lambda. We compare these alternatives using clients implemented with serverless AWS Lambda functions, and also a local machine, an Amazon virtual machine (VM), and a Google VM. Our data APIs accessed a managed Amazon Aurora PostgreSQL cluster populated with the U.S. Centers for Medicare & Medicaid Services (CMS) Open Payments dataset. Our GraphQL implementations show content-dependent performance compared to REST, with Apollo Server demonstrating 25-67% faster average round-trip times vs. REST for most operations, but worse scalability than REST with very high concurrent workloads. Our findings provide practical guidance for developers selecting serverless architectures for GraphQL and REST APIs based on specific application requirements, network conditions, and expected request volumes.

In-memory key-value stores are critical caching infrastructure for numerous cloud services. Unlike traditional CPU caches that often assume uniform item sizes, cloud caches frequently handle heterogeneous-sized objects, introducing significant challenges in cache management, particularly in shared multi-tenant environments. Existing cache sharing solutions designed for uniform-sized objects are often not optimized for these scenarios.This paper presents a probabilistic cache sharing scheme that dynamically adapts eviction probabilities across different traffic classes based on their performance. Our approach redistributes cache space at eviction events, reclaiming space probabilistically from one class to serve the needs of another. Our scheme operates independently of the underlying per-class eviction policies and adapts to time-varying traffic patterns. We evaluate our approach using real-world traces with heterogeneous object sizes, demonstrating its effectiveness in dynamically allocating cache space and improving overall cache performance in shared environments.

Modern Software as a Service (SaaS) platforms, such as the Microsoft Power Platform, simplify maintenance efforts through abstraction of underlying layers such as managing server infrastructure or OS-level configurations. Nevertheless, they still allow custom code extensions to extend and customize the application behavior. Yet, this implies restricted control and limited insights with respect to performance.Often, built-in performance reportings do not provide enough detail for efficient problem root cause analysis. Open standards, such as OpenTelemetry, make it easier to implement a more advanced aligned tracing ecosystem. Yet, while monitoring on traditional, non-cloud systems has been thoroughly studied, collecting telemetry data on SaaS systems faces many more complex challenges such as sandboxing, e.g., limited API access, or collecting end-to-end traces through multiple distributed entry points. Besides platform experience, deep technical knowledge about serialization, transmission, and caching costs is essential to understand the process and assess the performance overhead introduced by collecting telemetry data.In this work, we discuss common challenges regarding SaaS observability and the performance impact that telemetry data collection introduces. For this, we modify Intermediate Language (IL) code, i.e., already compiled code, on the Microsoft Power Platform to collect telemetry data for each method execution. A key finding is that a significant and often unreported portion of performance overhead originates from the SaaS platform itself, a crucial distinction from the overhead introduced by telemetry collection. Finally, we propose a concept for adaptive telemetry collection based on execution plan prediction, which aims to balance observability detail with performance overhead by estimating monitoring costs.

We analyze a recently published dataset of 396 real-world cloud architectures deployed on AWS, from companies belonging to a wide range of industries. From this dataset, we identify those architectures that contain HPC or edge components and characterize their designs. Specifically, we investigate the prevalence and interplay of AWS services within these architectures, examine the types of storage systems employed, assess architectural complexity and the use of machine learning services, discuss the implications of our findings and how representative these results are of HPC and edge architectures in the cloud. This characterization provides valuable insights into current industry practices and trends in building robust and scalable HPC and edge solutions in the cloud continuum, and can be valuable for those seeking to better understand how these architectures are being built and to guide new research.

Serverless Function-as-a-Service (FaaS) is a popular cloud paradigm to quickly and cheaply implement complex applications. Because the function instances cloud providers start to execute user code run on shared infrastructure, their performance can vary. From a user perspective, slower instances not only take longer to complete, but also increase cost due to the pay-per-use model of FaaS services where execution duration is billed with microsecond accuracy. In this paper, we present MINOS, a system to take advantage of this performance variation by intentionally terminating instances that are slow. Fast instances are not terminated, so that they can be re-used for subsequent invocations. One use case for this are data processing and machine learning workflows, which often download files as a first step, during which MINOS can run a short benchmark. Only if the benchmark passes, the main part of the function is actually executed. Otherwise, the request is re-queued and the instance crashes itself, so that the platform has to assign the request to another (potentially faster) instance. In our experiments, this leads to a speedup of up to 13% in the resource intensive part of a data processing workflow, resulting in up to 4% faster overall performance (and consequently 4% cheaper prices). Longer and complex workflows lead to increased savings, as the pool of fast instances is re-used more often. For platforms exhibiting this behavior, users get better performance and save money by wasting more of the platforms resources.

Cloud-native deployments are typically ad-hoc and fragmented, leading to repeated misconfigurations and increased security risk. A challenge that arises often in modern deployment processes is the use of diverse heterogeneous technologies, such as Infrastructure-as-Code frameworks and cloud-specific tools. This causes a technological lock-in, making migration, adoption of alternative solutions, and supporting multi-cloud environments difficult. Another issue that further complicates current deployment processes is the collaboration of multiple actors during the process, such as developers, cloud architects, and business decision-makers whose roles are not well defined.To address these challenges, we advocate for a constraint-based approach of deployment. We emphasize security and compliance where constraints are declarative and treated as first-class inputs. This process would enable the specification of high-level constraints (e.g. security, cost, legal, etc.) and technical requirements in a unified and structured manner. This makes deployments portable across environments and cloud providers, auditable, and resilient to evolution. By introducing clear role separation and constraints validation, our approach promotes accountability while reducing the cognitive load on individual deployment actors.We propose a framework that enables actors involved in the deployment to specify constraints at various stages (architecture level, implementation level, etc.). These constraints are evaluated and verified by a constraint solver, and results into a set of feasible deployment options. Deployment scripts are generated from one of the feasible deployments in order to deploy the application. It is worth stressing that our intention is not to replace existing deployment workflows, but rather to provide a complementary approach.