【正版 】流式系统 Sreaming Sysems 数据系统处理 流式作业和批处理作业的正确性 流式批量数据处理模式书籍东南大学出版ttx pdf snb 115盘 kindle 在线 下载 pmlz mobi

在传统的数据处理流程中，总是先收集数据，然后将数据放到DB中。当人们需要的时候通过DB对数据做query，得到答案或进行相关的处理。这样看起来虽然非常合理，但是结果却非常的紧凑，尤其是在一些实时搜索应用环境中的某些具体问题，类似于MapReduce方式的离线处理并不能很好地解决问题。这就引出了一种新的数据计算结构---流计算方式。它可以很好地对大规模流动数据在不断变化的运动过程中实时地进行分析，捕捉到可能有用的信息，并把结果发送到下一计算节点。本书讲解流计算原理。

Preface Or: What Are You Getting Yourself Into Here?

Part Ⅰ.The Beam Model

1.Streaming 101

Terminology: What Is Streaming?

On the Greatly Exaggerated Limitations of Streaming

Event Time Versus Processing Time

Data Processing Patterns

Bounded Data

Unbounded Data: Batch

Unbounded Data: Streaming

Summary

2.The What, Where, When, and How of Data Processing

Roadmap

Batch Foundations: What and Where

What: Transformations

Where: Windowing

Going Streaming: When and How

When: The Wonderful Thing About Triggers Is Triggers Are Wonderful Things!

When: Watermarks

When: Early/On-Time~Late Triggers FTWI

When: Allowed Lateness (i.e., Garbage Collection

How: Accumulation

Summary

3.Watermarks

Definition

Source Watermark Creation

Perfect Watermark Creation

Heuristic Watermark Creation

Watermark Propagation

Understanding Watermark Propagation

Watermark Propagation and Output Timestamps

The Tricky Case of Overlapping Windows

Percentile Watermarks

Processing-Time Watermarks

Case Studies

Case Study: Watermarks in Google Cloud Dataflow

Case Study: Watermarks in Apache Flink

Case Study: Source Watermarks for Google Cloud Pub/Sub

Summary

4.Advanced Windowing

When/Where: Processing-Time Windows

Event-Time Windowing

Processing-Time Windowing via Triggers

Processing-Time Windowing via Ingress Time

Where: Session Windows

Where: Custom Windowing

Variations on Fixed Windows

Variations on Session Windows

One Size Does Not Fit All

Summary

5.Exactly-Once and Side Effects

Why Exactly Once Matters

Accuracy Versus Completeness

Side Effects

Problem Definition

Ensuring Exactly Once in Shuffle

Addressing Determinism

Performance

Graph Optimization

Bloom Filters

Garbage Collection

Exactly Once in Sources

Exactly Once in Sinks

Use Cases

Example Source: Cloud Pub/Sub

Example Sink: Files

Example Sink: Google BigQuery

Other Systems

Apache Spark Streaming

Apache Flink

Summary

Part Ⅱ.Streams and Tables

6.Streams and Tables

Stream-and-Table Basics Or: a Special Theory of Stream and Table Relativity

Toward a General Theory of Stream and Table Relativity

Batch Processing Versus Streams and Tables

A Streams and Tables Analysis of MapReduce

Reconciling with Batch Processing

What, Where, When, and How in a Streams and Tables World

What: Transformations

Where: Windowing

When: Triggers

How: Accumulation

A Holistic View Of Streams and Tables in the Beam Model

A General Theory of Stream and Table Relativity

Summary

7.The Practicalities of Persistent State

Motivation

The Inevitability of Failure

Correctness and Efficiency

Implicit State

Raw Grouping

Incremental Combining

Generalized State

Case Study: Conversion Attribution

Conversion Attribution with Apache Beam

Summary

8.Streaming SQL

What Is Streaming SQL?

Relational Algebra

Time-Varying Relations

Streams and Tables

Looking Backward: Stream and Table Biases

The Beam Model: A Stream-Biased Approach

The SQL Model: A Table-Biased Approach

Looking Forward: Toward Robust Streaming SQL

Stream and Table Selection

Temporal Operators

Summary

9.Streaming Joins

All Your loins Are Belong to Streaming

Unwindowed loins

FULL OUTER

LEFT OUTER

RIGHT OUTER

INNER

ANTI

SEMI

Windowed loins

Fixed Windows

Temporal Validity

Summary

10.The Evolution of Large-Scale Data Processing

MapReduce

Hadoop

Flume

Storm

Spark

MillWheel

Kafka

Cloud Dataflow

Flink

Beam

Summary

Index

Tyler Akidau，是Google的高级软件工程师，担任着Data Processing Languages & Systems小组技术负责人的职务。他也是Apache Beam PMC的创始成员。

Slava Chernyak，是Google的高级软件工程师。他花了六年时间研究Google内部的大规模流式数据处理系统。

Reuven Lax，是Google的高级软件工程师，在过去十年间一直在帮助制定Google的数据处理和分析策略，同时他也是Apache Beam PMC的成员。

暂无出版社相关信息，正在全力查找中！

暂无相关书籍摘录，正在全力查找中！

在线阅读地址：【正版 】流式系统 Sreaming Sysems 数据系统处理 流式作业和批处理作业的正确性 流式批量数据处理模式书籍东南大学出版ttx在线阅读

在线听书地址：【正版 】流式系统 Sreaming Sysems 数据系统处理 流式作业和批处理作业的正确性 流式批量数据处理模式书籍东南大学出版ttx在线收听

在线购买地址：【正版 】流式系统 Sreaming Sysems 数据系统处理 流式作业和批处理作业的正确性 流式批量数据处理模式书籍东南大学出版ttx在线购买

In a healthy pipeline, most arriving records will not be duplicates. We can use that fact to greatly improve performance via Bloom filters, which are compact data structures that allow for quick set-membership checks. Bloom filters have a very interesting property: they can return false positives but never false negatives. If the filter says “Yes, the element is in the set,” we know that the element is probably in the set (with a probability that can be calculated). However, if the filter says an element is not in the set, it definitely isn’t. This function is a perfect fit for the task at hand.

书籍介绍

在传统的数据处理流程中，总是先收集数据，然后将数据放到DB中。当人们需要的时候通过DB对数据做query，得到答案或进行相关的处理。这样看起来虽然非常合理，但是结果却非常的紧凑，尤其是在一些实时搜索应用环境中的某些具体问题，类似于MapReduce方式的离线处理并不能很好地解决问题。这就引出了一种新的数据计算结构---流计算方式。它可以很好地对大规模流动数据在不断变化的运动过程中实时地进行分析，捕捉到可能有用的信息，并把结果发送到下一计算节点。本书讲解流计算原理。