Fixing things that were broken in a merge

contents/core/ml_systems/ml_systems.qmd

@@ -33,11 +33,11 @@ The deployment of machine learning systems across varied computing environments
 
 ## Overview
 
-ML is rapidly evolving, with new paradigms reshaping how models are developed, trained, and deployed. The field is experiencing significant innovation driven by advancements in hardware, software, and algorithmic techniques. These developments are enabling machine learning to be applied in diverse settings, from large-scale cloud infrastructures to edge devices and even tiny, resource-constrained environments.
-
 Modern machine learning systems span a spectrum of deployment options, each with its own set of characteristics and use cases. At one end, we have cloud-based ML, which leverages powerful centralized computing resources for complex, data-intensive tasks. Moving along the spectrum, we encounter edge ML, which brings computation closer to the data source for reduced latency and improved privacy. Mobile ML further extends these capabilities to smartphones and tablets, while at the far end, we find Tiny ML, which enables machine learning on extremely low-power devices with severe memory and processing constraints.
 
-This chapter explores the landscape of contemporary machine learning systems, covering four key approaches: Cloud ML, Edge ML, Mobile ML, and Tiny ML. @fig-cloud-edge-TinyML-comparison illustrates the spectrum of distributed intelligence across these approaches, providing a visual comparison of their characteristics. We will examine the unique characteristics, advantages, and challenges of each approach, as depicted in the figure. Additionally, we will discuss the emerging trends and technologies that are shaping the future of machine learning deployment, considering how they might influence the balance between these three paradigms.
+This spectrum of deployment can be visualized like Earth's geological features, each operating at different scales in our computational landscape. Cloud ML systems operate like continents, processing vast amounts of data across interconnected centers; Edge ML exists where these continental powers meet the sea, creating dynamic coastlines where computation flows into local waters; Mobile ML moves through these waters like ocean currents, carrying computing power across the digital seas; and where these currents meet the physical world, TinyML systems rise like islands, each a precise point of intelligence in the vast computational ocean.
+
+@fig-cloud-edge-TinyML-comparison illustrates the spectrum of distributed intelligence across these approaches, providing a visual comparison of their characteristics. We will examine the unique characteristics, advantages, and challenges of each approach, as depicted in the figure. Additionally, we will discuss the emerging trends and technologies that are shaping the future of machine learning deployment, considering how they might influence the balance between these three paradigms.
 
 ![Cloud vs. Edge vs. Mobile vs. Tiny ML: The Spectrum of Distributed Intelligence. Source: ABI Research -- Tiny ML.](images/png/cloud-edge-tiny.png){#fig-cloud-edge-TinyML-comparison}
 
@@ -95,7 +95,15 @@ This progression reflects a broader trend in computing towards more distributed,
 
 ## Cloud ML
 
-Cloud Machine Learning (Cloud ML) is a subfield of machine learning that leverages the power and scalability of cloud computing infrastructure to develop, train, and deploy machine learning models. By utilizing the vast computational resources available in the cloud, Cloud ML enables the efficient handling of large-scale datasets and complex machine learning algorithms. @fig-cloud-ml provides an overview of Cloud ML's capabilities which we will discuss in greater detail throughout this section.
+The vast computational demands of modern machine learning often require the scalability and power of centralized cloud infrastructures. Cloud Machine Learning (Cloud ML) handles tasks such as large-scale data processing, collaborative model development, and advanced analytics. Cloud data centers leverage distributed architectures, offering specialized resources to train complex models and support diverse applications, from recommendation systems to natural language processing.
+
+::: {.callout-note}
+### Definition of Cloud ML
+
+**Cloud Machine Learning (Cloud ML)** refers to the deployment of machine learning models on *centralized computing infrastructures*, such as data centers. These systems operate in the *kilowatt to megawatt* power range and utilize *specialized computing systems* to handle *large-scale datasets* and train *complex models*. Cloud ML offers *scalability* and *computational capacity*, making it well-suited for tasks requiring extensive resources and collaboration. However, it depends on *consistent connectivity* and may introduce *latency* for real-time applications.
+:::
+
+@fig-cloud-ml provides an overview of Cloud ML's capabilities, which we will discuss in greater detail throughout this section.
 
 ![Section overview for Cloud ML.](images/png/cloudml.png){#fig-cloud-ml fig-pos='!t'}
 
@@ -203,7 +211,17 @@ Cloud ML plays a role in bolstering user security by powering anomaly detection
 
 ## Edge ML
 
-Edge Machine Learning (Edge ML) runs machine learning algorithms directly on endpoint devices or closer to where the data is generated rather than relying on centralized cloud servers. This approach brings computation closer to the data source, reducing the need to send large volumes of data over networks, often resulting in lower latency and improved data privacy. @fig-edge-ml provides an overview of this section.
+As machine learning applications grow, so does the need for faster, localized decision-making. Edge Machine Learning (Edge ML) shifts computation away from centralized servers, processing data closer to its source. This paradigm is critical for time-sensitive applications, such as autonomous systems, industrial IoT, and smart infrastructure, where minimizing latency and preserving data privacy are paramount. Edge devices, like gateways and IoT hubs, enable these systems to function efficiently while reducing dependence on cloud infrastructures.
+
+::: {.callout-note}
+### Definition of Edge ML
+
+**Edge Machine Learning (Edge ML)** describes the deployment of machine learning models at or near the *edge of the network*[^fn-edge-network]. These systems operate in the *tens to hundreds of watts* range and rely on *localized hardware* optimized for *real-time processing*. Edge ML minimizes *latency* and enhances *privacy* by processing data locally, but its primary limitation lies in *restricted computational resources*.
+:::
+
+[^fn-edge-network]: The "edge of the network" refers to devices or systems positioned between centralized cloud infrastructures and end-user devices, such as gateways, IoT hubs, or industrial sensors.
+
+@fig-edge-ml provides an overview of this section.
 
 ![Section overview for Edge ML.](images/png/edgeml.png){#fig-edge-ml  fig-pos='!t'}
 
@@ -267,7 +285,13 @@ The applicability of Edge ML is vast and not limited to these examples. Various
 
 ## Mobile ML
 
-Mobile Machine Learning (Mobile ML) represents a specialized branch of Edge ML that focuses on deploying and running machine learning models directly on mobile devices like smartphones and tablets. This approach leverages the computational capabilities of modern mobile processors to perform ML tasks locally, offering a balance between the power of edge computing and the ubiquity of personal devices.
+Machine learning is increasingly being integrated into portable devices like smartphones and tablets, empowering users with real-time, personalized capabilities. Mobile Machine Learning (Mobile ML) supports applications like voice recognition, computational photography, and health monitoring, all while maintaining data privacy through on-device computation. These battery-powered devices are optimized for responsiveness and can operate offline, making them indispensable in everyday consumer technologies.
+
+::: {.callout-note}
+### Definition of Mobile ML
+
+**Mobile Machine Learning (Mobile ML)** enables machine learning models to run directly on *portable, battery-powered devices* like smartphones and tablets. Operating within the *single-digit to tens of watts* range, Mobile ML leverages *on-device computation* to provide *personalized and responsive applications*. This paradigm preserves *privacy* and ensures *offline functionality*, though it must balance *performance* with *battery and storage limitations*.
+:::
 
 ### Characteristics
 
@@ -329,7 +353,15 @@ These applications demonstrate how Mobile ML bridges the gap between cloud-based
 
 ## Tiny ML
 
-Tiny ML sits at the crossroads of embedded systems and machine learning, representing a burgeoning field that brings smart algorithms directly to tiny microcontrollers and sensors. These microcontrollers operate under severe resource constraints, particularly regarding memory, storage, and computational power. @fig-tiny-ml encapsulates the key aspects of Tiny ML discussed in this section.
+Tiny Machine Learning (Tiny ML) brings intelligence to the smallest devices, from microcontrollers to embedded sensors, enabling real-time computation in resource-constrained environments. These systems power applications such as predictive maintenance, environmental monitoring, and simple gesture recognition. Tiny ML devices are optimized for energy efficiency, often running for months or years on limited power sources, such as coin-cell batteries, while delivering actionable insights in remote or disconnected environments.
+
+::: {.callout-note}
+### Definition of Tiny ML
+
+**Tiny Machine Learning (Tiny ML)** refers to the execution of machine learning models on *ultra-constrained devices*, such as microcontrollers and sensors. These devices operate in the *milliwatt to sub-watt* power range, prioritizing *energy efficiency* and *compactness*. Tiny ML enables *localized decision-making* in resource-constrained environments, excelling in applications where *extended operation on limited power sources* is required. However, it is limited by *severely restricted computational resources*.
+:::
+
+@fig-tiny-ml encapsulates the key aspects of Tiny ML discussed in this section. 
 
 ![Section overview for Tiny ML.](images/png/tinyml.png){#fig-tiny-ml}
 
@@ -406,7 +438,13 @@ In summary, Tiny ML serves as a trailblazer in the evolution of machine learning
 
 ## Hybrid ML
 
-Systems architects rarely confine themselves to a single approach, instead combining various paradigms to create more nuanced solutions. These "Hybrid ML" approaches leverage the complementary strengths we've analyzed---from cloud's computational power to tiny's efficiency---while mitigating their individual limitations. Architects create new architectural patterns that balance competing demands for performance, privacy, and resource efficiency, opening up possibilities for more sophisticated ML applications that better meet complex real-world requirements.
+The increasingly complex demands of modern applications often require a blend of machine learning approaches. Hybrid Machine Learning (Hybrid ML) combines the computational power of the cloud, the efficiency of edge and mobile devices, and the compact capabilities of Tiny ML. This approach enables architects to create systems that balance performance, privacy, and resource efficiency, addressing real-world challenges with innovative, distributed solutions.
+
+::: {.callout-note}
+### Definition of Hybrid ML
+
+**Hybrid Machine Learning (Hybrid ML)** refers to the integration of multiple ML paradigms—such as Cloud, Edge, Mobile, and Tiny ML—to form a unified, distributed system. These systems leverage the *complementary strengths* of each paradigm while addressing their *individual limitations*. Hybrid ML supports *scalability, adaptability,* and *privacy-preserving capabilities,* enabling sophisticated ML applications for diverse scenarios. By combining centralized and decentralized computing, Hybrid ML facilitates efficient resource utilization while meeting the demands of complex real-world requirements.
+:::
 
 ### Design Patterns
 
@@ -422,7 +460,7 @@ One of the most common hybrid patterns is the train-serve split, where model tra
 
 Hierarchical processing creates a multi-tier system where data and intelligence flow between different levels of the ML stack. In industrial IoT applications, tiny sensors might perform basic anomaly detection, edge devices aggregate and analyze data from multiple sensors, and cloud systems handle complex analytics and model updates. For instance, we might see ESP32-CAM devices performing basic image classification at the sensor level with their minimal 520KB RAM, feeding data up to Jetson AGX Orin devices for more sophisticated computer vision tasks, and ultimately connecting to cloud infrastructure for complex analytics and model updates.
 
-This hierarchy allows each tier to handle tasks appropriate to its capa­bil­ities---Tiny ML devices handle immediate, simple decisions; edge devices manage local coordination; and cloud systems tackle complex analytics and learning tasks. Smart city installations often use this pattern, with street-level sensors feeding data to neighborhood-level edge processors, which in turn connect to city-wide cloud analytics.
+This hierarchy allows each tier to handle tasks appropriate to its capabilities---Tiny ML devices handle immediate, simple decisions; edge devices manage local coordination; and cloud systems tackle complex analytics and learning tasks. Smart city installations often use this pattern, with street-level sensors feeding data to neighborhood-level edge processors, which in turn connect to city-wide cloud analytics.
 
 #### Progressive Deployment
 
@@ -488,7 +526,9 @@ Edge ML shifts the focus to local processing, prioritizing inference capabilitie
 
 Mobile ML leverages the capabilities of personal devices, particularly smartphones and tablets. With specialized hardware like Apple's A17 Pro chip, mobile implementations enable sophisticated ML capabilities while maintaining user privacy and providing offline functionality. This paradigm has revolutionized applications from computational photography to on-device speech recognition.
 
-Tiny ML represents the frontier of embedded ML, bringing intelligence to highly constrained devices. Operating on microcontrollers like the Arduino Nano 33 BLE Sense, tiny implementations must carefully balance functionality with severe resource constraints. Despite these limitations, Tiny ML enables ML capabilities in scenarios where power efficiency and size constraints are paramount.
+Tiny ML represents the frontier of embedded ML, bringing intelligence to highly constrained devices. Operating on microcontrollers like the Arduino Nano 33 BLE Sense[^fn-arduino], tiny implementations must carefully balance functionality with severe resource constraints. Despite these limitations, Tiny ML enables ML capabilities in scenarios where power efficiency and size constraints are paramount.
+
+[^fn-arduino]: The Arduino Nano 33 BLE Sense, introduced in 2019, is a microcontroller specifically designed for Tiny ML applications, featuring sensors and Bluetooth connectivity to facilitate on-device intelligence.  
 
 ### System Principles Layer
 
@@ -579,6 +619,26 @@ Cost structures differ markedly as well. Cloud ML typically involves ongoing ope
 
 These comparisons reveal that each paradigm has distinct advantages and limitations. Cloud ML excels at complex, data-intensive tasks but requires constant connectivity. Edge ML offers a balance of computational power and local processing. Mobile ML provides personalized intelligence on ubiquitous devices. TinyML enables ML in previously inaccessible contexts but requires careful optimization. Understanding these trade-offs is crucial for selecting the appropriate deployment strategy for specific applications and constraints.
 
+## ML Deployment Decision Framework
+
+We have examined the diverse paradigms of machine learning systems—Cloud ML, Edge ML, Mobile ML, and Tiny ML—each with its own characteristics, trade-offs, and use cases. Selecting an optimal deployment strategy requires careful consideration of multiple factors.
+
+To facilitate this decision-making process, we present a structured framework in @fig-mlsys-playbook-flowchart. This framework distills the chapter's key insights into a systematic approach for determining the most suitable deployment paradigm based on specific requirements and constraints.
+
+![A decision flowchart for selecting the most suitable ML deployment paradigm.](images/png/mlsys_playbook.png){#fig-mlsys-playbook-flowchart}
+
+The framework is organized into five fundamental layers of consideration:
+
+- **Privacy**: Determines whether processing can occur in the cloud or must remain local to safeguard sensitive data.
+- **Latency**: Evaluates the required decision-making speed, particularly for real-time or near-real-time processing needs.
+- **Reliability**: Assesses network stability and its impact on deployment feasibility.
+- **Compute Needs**: Identifies whether high-performance infrastructure is required or if lightweight processing suffices.
+- **Cost and Energy Efficiency**: Balances resource availability with financial and energy constraints, particularly crucial for low-power or budget-sensitive applications.
+
+As designers progress through these layers, each decision point narrows the viable options, ultimately guiding them toward one of the four deployment paradigms. This systematic approach proves valuable across various scenarios. For instance, privacy-sensitive healthcare applications might prioritize local processing over cloud solutions, while high-performance recommendation engines typically favor cloud infrastructure. Similarly, applications requiring real-time responses often gravitate toward edge or mobile-based deployment.
+
+While not exhaustive, this framework provides a practical roadmap for navigating deployment decisions. By following this structured approach, system designers can evaluate trade-offs and align their deployment choices with technical, financial, and operational priorities, even as they address the unique challenges of each application.
+
 ## Conclusion
 
 This chapter has explored the diverse landscape of machine learning systems, highlighting their unique characteristics, benefits, challenges, and applications. Cloud ML leverages immense computational resources, excelling in large-scale data processing and model training but facing limitations such as latency and privacy concerns. Edge ML bridges this gap by enabling localized processing, reducing latency, and enhancing privacy. Mobile ML builds on these strengths, harnessing the ubiquity of smartphones to provide responsive, user-centric applications. At the smallest scale, Tiny ML extends the reach of machine learning to resource-constrained devices, opening new domains of application.