Graph

A computational graph is a series of TensorFlow operations arranged into a graph. The graph is composed of two types of objects.

tf.Operation (or “ops”): The nodes of the graph. Operations describe calculations that consume and produce tensors.
tf.Tensor: The edges in the graph. These represent the values that will flow through the graph. Most TensorFlow functions return tf.Tensors.

Quoting from 1

What is a tf.Graph?
A tf.Graph contains two relevant kinds of information:

Graph structure. The nodes and edges of the graph, indicating how individual operations are composed together, but not prescribing how they should be used. The graph structure is like assembly code: inspecting it can convey some useful information, but it does not contain all of the useful context that source code conveys.

Graph collections. TensorFlow provides a general mechanism for storing collections of metadata in a tf.Graph. The tf.add_to_collection function enables you to associate a list of objects with a key (where tf.GraphKeys defines some of the standard keys), and tf.get_collection enables you to look up all objects associated with a key. Many parts of the TensorFlow library use this facility: for example, when you create a tf.Variable, it is added by default to collections representing “global variables” and “trainable variables”. When you later come to create a tf.train.Saver or tf.train.Optimizer, the variables in these collections are used as the default arguments.

Session

Graph && Session

Graph是用来定义操作和变量的. 定义好了之后, 交给Session进行加载和计算. Session同时管理资源.

References

TensorFlow – Guide – Low Level APIs – Introduction

TensorFlow – Guide – Low Level APIs – Graph and Sessions

简介

关注点分离(SoC - Separation of Concerns)原则是软件工程相关的设计原则之一. SoC原则的本质是将应用切成不同的部分, 每个部分解决一个单独的关注点.

MVC模式代表的是Model-View-Controller模式, 分别代表了三种角色, 模型, 视图和控制器, 该模式是应用到面向对象编程的SoC原则.

模型是核心的部分, 代表着应用的信息本源, 包含和管理(业务)逻辑, 数据, 状态及应用的规则.

视图是模型的可视化表现, 举例来说, 程序的图形化界面, 终端文本输出, 智能手机的应用图形界面, 各种类型的图(柱形图, 饼状图等). 视图只是展示数据, 并不处理数据.

控制器是模型与视图之间的链接. 模型和视图之间的所有通信都是通过控制器进行控制.

参考 : 精通Python设计模式 – 第八章 模型-视图-控制器模式

References

MVC 模式 | 菜鸟教程
精通Python设计模式 – 第八章 模型-视图-控制器模式
MVC简介
MVC框架 – 百度百科

Introduction

ROC

ROC的全拼是 Receiver Operating Characteristic 曲线, 中文名是”受试者工作特征”.
ROC曲线的纵轴是 TPR (True Positive Rate), 而横轴是 FPR (False Positive Rate). 这两者的定义可以参考文章混淆矩阵 Confusion Matrix

在ROC曲线图中, 对角线对应”随机猜想”模型, 是因为在”随机猜想”模型中, 任何一个样例都有50%的概率被猜对.
而点(0,1)则对应于将所有正例排在所有反例之前的”理想模型”.

AUC

AUC 的全拼为 Area Under ROC Curve.

Implementation

References

机器学习(西瓜书) Section 2.3.3

DRFAT

假设我们需要在Python中下载一个文件, 其对应url地址为 URL= 'http://***/test/demo.zip'

使用urllib进行下载

import urllib
urllib.urlretrieve(URL, "demo.zip")

使用urllib2进行下载

import urllib2
f = urllib2.urlopen(URL)
data = f.read()
with open("demo2.zip", "wb") as code:
    code.write(data)

使用requests进行下载

import requests
r = requests.get(URL)
with open("demo3.zip", "wb") as code:
    code.write(r.content)

在HTTP相关处理中使用Python是不必要的麻烦, 这包括urllib2模块以巨大的复杂性代价获取综合性的功能。相比于urllib2, Requests模块更能简约的支持完整的简单用例。

举一个简单的例子来说明两者之间的差别. 假设现在需要从URL 获取资源并且查看返回代码，content-type头信息，还有response的主体内容. 分别使用urllib2和Requests进行实现.

>>> import urllib2
>>> response = urllib2.urlopen(URL)
>>> response.getcode()
200
>>> response.headers.getheader('content-type')
'text/html; charset=utf-8'
>>> response.read()
'Hello, world!'

>>> import requests
>>> response = requests.get(URL)
>>> response.status_code
200
>>> response.headers['content-type']
'text/html; charset=utf-8'
>>> response.content
u'Hello, world!

这两种方法很相似, 相对于urllib2调用方法读取response中的属性信息, Requests则是使用属性名来获取对应的属性值.
两者还有两个细微但是很重要的差别:
1. Requests 自动的把返回信息用Unicode解码, 上面例子中可以看出, response.content返回的是unicode对象.
2. Requests 自动保存了返回内容, 所以你可以读取多次, 而不像urllib2.urlopen()那样返回的只是一个类似文件类型只能读取一次的对象。

第二点是在python交互式环境下操作代码很令人讨厌的事情

一个复杂一点的例子：现在让我们尝试下复杂点得例子：使用GET方法获取 http://foo.test/secret 的资源，这次需要基本的http验证。使用上面的代码作为模板，好像我们只要把urllib2.urlopen() 到requests.get()之间的代码换成可以发送username，password的请求就行了

这是urllib2的方法：

>>> import urllib2
>>> password_manager = urllib2.HTTPPasswordMgrWithDefaultRealm()
>>> password_manager.add_password(None, url, 'dan', 'h0tdish')
>>> auth_handler = urllib2.HTTPBasicAuthHandler(password_manager)
>>> opener = urllib2.build_opener(auth_handler)
>>> urllib2.install_opener(opener)
>>> response = urllib2.urlopen(url)
>>> response.getcode()
200
>>> response.read()
'Welcome to the secret page!'

一个简单的方法中实例化了HTTPPasswordMgrWithDefaultRealm和HTTPBasicAuthHandler，然后组建了第三个类opener，最后还要装载到全局的urllib2模块中，最后才调用了urlopen.

那Requests是怎么样解决同样的问题的呢？

>>> import requests
>>> response = requests.get(URL, auth=('dan', 'h0tdish'))
>>> response.status_code
200
>>> response.content
u'Welcome to the secret page!'

只是在调用方法的时候增加了一个auth关键字函数.

错误处理 Error HandlingRequests 对错误的处理也是很非常方面。如果你使用了不正确的用户名和密码，urllib2会引发一个urllib2.URLError错误，然而Requests会像你期望的那样返回一个正常的response对象。只需查看response.ok的布尔值便可以知道是否登陆成功。

>>> response = requests.get(URL, auth=('dan', 'wrongPass'))
>>> response.ok
False

推荐使用Requests来进行Http请求, 包括文件下载.

python下载文件的三种方法
urllib2 in Py2
urllib in Py3
python的扩展包requests的高级用法
Python urllib的urlretrieve()函数解析 这里提到了retrieval中的回调, 可以显示下载进度, 很有意思, 没事可以看一下.
How do I download a file over HTTP using Python?

Pickle在处理大小超过4G的文件时会抛出异常:OverflowError: cannot serialize a bytes object larger than 4 GiB

---

Stack Overflow 上的回答是: 在Python 3.4之后不在有这个限制, 但是需要声明使用的协议等级.

Not anymore in Python 3.4 which has PEP 3154 and Pickle 4.0
https://www.python.org/dev/peps/pep-3154/

But you need to say you want to use version 4 of the protocol:
https://docs.python.org/3/library/pickle.html

pickle.dump(d, open("file", 'w'), protocol=4)

Quoting From Eric Levieil

这里补充一点, 可以通过pickle.dumps(obj=OBJ,protocal=pickle.HIGHEST_PROTOCOL)直接指定最高等级的 Protocol.

还有一个不是那么优雅的方案, 即先将对象转换为string类型, 然后每次只通过pickle处理不超过4G的数据.

def dump_over_4g(data, file_path, max_bytes=2 ** 31 - 1):
    """ 将超过4g大小的对象写入本地(mac os 系统上pickle处理超过4g的文件会抛出异常) 2**31 - 1 代表 4g """
    bytes_out = pickle.dumps(data)
    with open(file_path, 'wb+') as w_file:
        for idx in range(0, len(bytes_out), max_bytes):
            w_file.write(bytes_out[idx:idx + max_bytes])

def read_over_4g(file_path, max_bytes=2 ** 31 - 1, mode='rb'):
    """ 读取超过大小超过4g的对象 """
    bytes_in = bytearray(0)
    input_size = os.path.getsize(file_path)
    with open(file_path, mode) as r_file:
        for _ in range(0, input_size, max_bytes):
            bytes_in += r_file.read(max_bytes)
    data = pickle.loads(bytes_in)
    return data

1. ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
   安装tensorflow报ImportError: libcublas.so.9.0: cannot open shared object file的解决方法（2018年4月）

2. TensorFlow 识别不到GPU / TensorFlow GPU support
   通过代码

import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

可以看到目前设备对TensorFlow的支持. 以如下为例, 系统目前没有识别出安装的1080Ti显卡, 因此只能看到CPU.

正常识别出显卡的效果如下:

没有识别出显卡的原因有很多, 最为常见的是 TensorFlow 版本和cuDNN, Cuda Toolkit的版本不匹配, 或是安装的时候没有将路径添加到系统路径中导致无法找到.
可以通过查询经过测试的构建配置 / tested_build_configurations来获取TensorFlow, cuDNN, Cuda Toolkit版本对应关系.

Translate from original page : tfx-serving-architeture

---

Architecture 架构

TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out of the box integration with TensorFlow models, but can be easily extended to serve other types of models.

TensorFlow Serving 是一个为了工业环境而设计的, 服务于机器学习模型的高效, 灵活的服务系统. TensorFlow Serving 能够在保持同样的服务架构和API的情况下让部署新的算法和实验更加容易. TensorFlow Serving 提供了和利用TensorFlow构建的模型非常契合的交互方式, 同时也可以很方便的扩展到其他类型的模型.

Key Concepts 关键概念

To understand the architecture of TensorFlow Serving, you need to understand the following key concepts:

未来理解 TensorFlow Serving的架构, 你需要理解以下关键概念:

Servables

Servables are the central abstraction in TensorFlow Serving. Servables are the underlying objects that clients use to perform computation (for example, a lookup or inference).

Servables 是 TensorFlow Serving 中的关键抽象概念. Servables 是一种底层的对象, 从而客户端可以利用它们进行计算(例如, 查询或者推断).

The size and granularity of a Servable is flexible. A single Servable might include anything from a single shard of a lookup table to a single model to a tuple of inference models. Servables can be of any type and interface, enabling flexibility and future improvements such as:

Servable 的大小和粒度都是非常灵活的. 单个 Servable 可以包含查询表的一个分块, 或者单个模型, 或者一组推断模型. Servables可以是任何类型或者接口, 未来的改进如下:

• streaming results
• experimental APIs
• asynchronous modes of operation

Servables do not manage their own lifecycle.

Servables 不管理它们自己的声明周期.

Typical servables include the following:

经典的 Servables 包含如下:

• a TensorFlow SavedModelBundle (tensorflow::Session)
• a lookup table for embedding or vocabulary lookups

Servable Versions

TensorFlow Serving can handle one or more versions of a servable over the lifetime of a single server instance. This enables fresh algorithm configurations, weights, and other data to be loaded over time. Versions enable more than one version of a servable to be loaded concurrently, supporting gradual rollout and experimentation. At serving time, clients may request either the latest version or a specific version id, for a particular model.

TensorFlow Serving 可以在单个服务器实例上处理一个Servable的一个或多个版本. 这意味着可以随着时间的变化, 用户可以刷新算法配置, 权重, 或者加载其他数据. 可同时加载一个 Servable 的多个版本, 支持逐步推广和实验. 在服务时间内, 客户端可以请求最新的版本, 或者根据版本id请求特定的版本.

Servable Streams

A servable stream is the sequence of versions of a servable, sorted by increasing version numbers.

Servable Stream 是 Servable 依照版本号进行升序排列的版本序列.

Models

TensorFlow Serving represents a model as one or more servables. A machine-learned model may include one or more algorithms (including learned weights) and lookup or embedding tables.

TensorFlow Serving 将 model 表示为一个或多个 Servable. 一个机器学习模型可能会包含一个或多个算法(包括学习权重), 查询表和嵌入表等.

You can represent a composite model as either of the following:

你可以将一个复杂模型通过如下的方式进行表示:

• multiple independent servables 多个独立 Servable
• single composite servable 单个复杂 Servable

A servable may also correspond to a fraction of a model. For example, a large lookup table could be sharded across many TensorFlow Serving instances.

一个 Servable 同样也可以相当于是一个模型的一部分. 举例来说, 一个大型的查询表可以被分割并表示为多个 TensorFlow Serving 实例.

Loaders

Loaders manage a servable’s life cycle. The Loader API enables common infrastructure independent from specific learning algorithms, data or product use-cases involved. Specifically, Loaders standardize the APIs for loading and unloading a servable.

Loaders 管理 Servable 的生命周期. Loader API 建立了脱离特定学习算法, 数据和产品用户案例的, 通用的结构. 需要特别提出的是, Loader 将加载和卸载 Servable 的API进行了标准化.

Sources

Sources are plugin modules that find and provide servables. Each Source provides zero or more servable streams. For each servable stream, a Source supplies one Loader instance for each version it makes available to be loaded. (A Source is actually chained together with zero or more SourceAdapters, and the last item in the chain emits the Loaders.)

Sources 是用来寻找和提供 Servable 的插件模块. 每个 Source 都提供了零或多个 Servable Stream.

TensorFlow Serving’s interface for Sources can discover servables from arbitrary storage systems. TensorFlow Serving includes common reference Source implementations. For example, Sources may access mechanisms such as RPC and can poll a file system.

TensorFlow Serving 中 Source 的接口可以从任意存储系统中发现 Servable. TensorFlow Serving 包含了通用的 Source 实现. 例如, Source 可以访问类似RPC的机制并挂载一个文件系统.

Sources can maintain state that is shared across multiple servables or versions. This is useful for servables that use delta (diff) updates between versions.

Sources 可以在多个 Servable 或版本中维持状态. 这对于使用版本件不同进行更新的 Servable 非常有用.

Aspired Versions

Aspired versions represent the set of servable versions that should be loaded and ready. Sources communicate this set of servable versions for a single servable stream at a time. When a Source gives a new list of aspired versions to the Manager, it supercedes the previous list for that servable stream. The Manager unloads any previously loaded versions that no longer appear in the list.

Aspired Versions 表示的是可以被加载和读取的 Servable 版本的集合. Source 每次访问这个集合来来获取单个 Servable Stream. 当 Source 交给 Manager 一个新的关于 Aspired Version 的列表时, 它将取代之前的列表. Manager 卸载任何之前存在但现在不存在的版本.

See the advanced tutorial to see how version loading works in practice.

查看advanced tutorial 来获取关于实际应用中版本加载如何工作的详情.

Managers

Managers handle the full lifecycle of Servables, including:

Manager 负责处理 Servable 的全部生命周期, 例如

• loading Servables
• serving Servables
• unloading Servables

Managers listen to Sources and track all versions. The Manager tries to fulfill Sources’ requests, but may refuse to load an aspired version if, say, required resources aren’t available. Managers may also postpone an “unload”. For example, a Manager may wait to unload until a newer version finishes loading, based on a policy to guarantee that at least one version is loaded at all times.

Manger 监听 Source 并追踪所有的版本. Manager 尝试满足 Source 的请求, 但可能会拒绝加载一个条件不允许的 Aspired Version, 例如, 资源不存在. Manager 也可能会延迟卸载的操作. 例如, 一个基于任何时刻都必须保持至少有一个版本可用侧率的 Manager 可能会等到新的版本完成加载后再进行老版本的卸载.

TensorFlow Serving Managers provide a simple, narrow interface — GetServableHandle() — for clients to access loaded servable instances.

TensorFLow Serving Manager 提供了一个简单, 限定的接口 GetServableHandle() 让客户端去访问加载的 Servable 实例.

Core

Using the standard TensorFlow Serving APis, TensorFlow Serving Core manages the following aspects of servables:

使用标准的 TensroFlow Serving API, TensorFlow Serving Core 管理 Servable的以下方面:

• lifecycle 生命周期
• metrics 指标

TensorFlow Serving Core treats servables and loaders as opaque objects.

TensorFlow Serving Core 将 Servable 和 Loader 当做黑盒进行处理.

Life of a Servable

Broadly speaking:

大概来说:

1. Sources create Loaders for Servable Versions.
2. Source 创建 Loader 来获取 Servable Version.

3. Loaders are sent as Aspired Versions to the Manager, which loads and serves them to client requests.

4. Loader 被当做 Aspired Version 发送给 Manager, Manager 加载并使用 Loader 来处理客户端请求.

In more detail:

细节上来说:

1. A Source plugin creates a Loader for a specific version. The Loader contains whatever metadata it needs to load the Servable.
2. Source 插件为一个 Loader 创建一个指定的版本. Loader 包含了所有加载 Servable 所需的元数据.

3. The Source uses a callback to notify the Manager of the Aspired Version.

4. Source 通过回调函数通知 Manager Aspired Version的相关信息.

5. The Manager applies the configured Version Policy to determine the next action to take, which could be to unload a previously loaded version or to load the new version.

6. Manager 依照配置的版本策略决定该如何进行下一步操作,可能是卸载老版本, 或者加载新版本.

7. If the Manager determines that it’s safe, it gives the Loader the required resources and tells the Loader to load the new version.

8. 如果 Manager 判断当前环境安全, 则 Manager 赋予 Loader 必要的资源并通知 Loader 可以加载新版本.

9. Clients ask the Manager for the Servable, either specifying a version explicitly or just requesting the latest version. The Manager returns a handle for the Servable.

10. 客户端向 Manger 请求 Servable, 不管是指定版本或最新版本. Manager 返回该 Servable 的操作接口.

For example, say a Source represents a TensorFlow graph with frequently updated model weights. The weights are stored in a file on disk.

举例来说, 假设有一个 Source, 其表示一张经常更新模型权重的 TensorFlow 图. 权重数据存储在硬盘上的某一个文件里.

1. The Source detects a new version of the model weights. It creates a Loader that contains a pointer to the model data on disk.
2. Source 检测到模型权重的新版本后, 创建一个包含指向硬盘上模型数据的指针的 Loader.

3. The Source notifies the Dynamic Manager of the Aspired Version.

4. Source 通知 Dynamic Manager 相关 Aspired Version 信息.

5. The Dynamic Manager applies the Version Policy and decides to load the new version.

6. Dynamic Manager 依照版本策略决定是否/何时加载新版本.

7. The Dynamic Manager tells the Loader that there is enough memory. The Loader instantiates the TensorFlow graph with the new weights.

8. Dynamic 通知 Loader, 内存可用. Loader 利用新的权重数据进行 TensorFlow Graph的实例化.

9. A client requests a handle to the latest version of the model, and the Dynamic Manager returns a handle to the new version of the Servable.

10. 客户端请求模型最新版本的操作接口. Dynamic Manager 返回最新版本的操作接口.

Extensibility 扩展性

TensorFlow Serving provides several extension points where you can add new functionality.

TensorFlow Serving 提供了多个可扩展点, 方便用户添加新功能.

Version Policy 版本管理

Version Policies specify the sequence of version loading and unloading within a single servable stream.

版本管理明确了在单个 Servable Stream 中加载和卸载版本的顺序.

TensorFlow Serving includes two policies that accommodate most known use- cases. These are the Availability Preserving Policy (avoid leaving zero versions loaded; typically load a new version before unloading an old one), and the Resource Preserving Policy (avoid having two versions loaded simultaneously, thus requiring double the resources; unload an old version before loading a new one). For simple usage of TensorFlow Serving where the serving availability of a model is important and the resource costs low, the Availability Preserving Policy will ensure that the new version is loaded and ready before unloading the old one. For sophisticated usage of TensorFlow Serving, for example managing versions across multiple server instances, the Resource Preserving Policy requires the least resources (no extra buffer for loading new versions).

TensorFlow Serving 对于大部分的用户案例提供了两个版本管理策略, 分别是 Availability Preserving Policy (避免没有版本加载的情况; 通常先加载新版本后卸载旧版本), 和 Resource Preserving Policy (避免同时拥有两个可用版本, 也就是资源的双倍占用; 在加载新版本之前卸载就版本). 在模型的服务可用性较为重要, 资源消耗低的简单环境下, Availability Preserving Policy 可以保证新版本在老版本卸载之前被加载. 在复杂的环境下, 例如, 多个服务实例上管理多个版本, Resource Preserving Policy 需要的资源会更少(没有加载新版本的额外资源需求)

Source

New Sources could support new filesystems, cloud offerings and algorithm backends. TensorFlow Serving provides some common building blocks to make it easy & fast to create new sources. For example, TensorFlow Serving includes a utility to wrap polling behavior around a simple source. Sources are closely related to Loaders for specific algorithms and data hosting servables.

新的 Source 可以支持新的文件系统, 云服务和算法后端. TensorFlow Serving 提供了一些通用的 building block 用来更简单更快的创建 Source. 举例来说, TensorFlow Serving 提供了装饰挂载在一个简单 Source 的方法. Source 因为特定的算法和数据支持 Servable 和 Loader 关联密切.

See the Custom Source document for more about how to create a custom Source.

查看Custom Source获取更多自定义 Source 的详情.

Loaders

Loaders are the extension point for adding algorithm and data backends. TensorFlow is one such algorithm backend. For example, you would implement a new Loader in order to load, provide access to, and unload an instance of a new type of servable machine learning model. We anticipate creating Loaders for lookup tables and additional algorithms.

Loader 是用来添加算法和数据后端的扩展点. TensorFlow 就是这样的一个算法后端. 例如, 如果你为了加载,提供访问和卸载一个新的机器学习模型而实现一个新的 Loader. 我们预先为查询表和增加的算法创建 Loader.

See the Custom Servable document to learn how to create a custom servable.

查看Custom Servable获取更多如何自定义 Servable 的详情.

Batcher

Batching of multiple requests into a single request can significantly reduce the cost of performing inference, especially in the presence of hardware accelerators such as GPUs. TensorFlow Serving includes a request batching widget that lets clients easily batch their type-specific inferences across requests into batch requests that algorithm systems can more efficiently process. See the Batching Guide for more information.

在单次请求中汇聚多个请求形成 Batching 可以有效降低模型推断结果的资源消耗, 特别是在例如GPU之类的加速器面前. TensorFlow Serving 包含了一个批量请求装饰器, 可以让客户端更容易的将他们的独特类型的请求放在一起, 从而让算法系统能够更有效的处理. 查看Batching Guide获取更多详情.

DataFrame 根据某一列值进行排序

df.sort_values(“xxx”,inplace=True)

DataFrame 根据多列值进行排序

DataFrame 筛选某一列值中的前百分之X的记录

临时解决办法:
假设DataFrame为DFI(DataFrameInstance), 目标列为A, 筛选列A中值前25%的记录, 则先计算 goal_value = DFI[‘A’].quantile(.75) 计算出在75%位置的数值是多少, 然后通过 DFI[DFI[‘A’] > goal_value] 来达到效果.

遍历 DataFrame

DataFrame.iterrows() 返回的结果是index和row的tuple. 其中row是Series类型.

在 Python深度学习 一书中7.33节中提高到了
为了找到一组更好的集成权重, 你可以使用随机搜索或简单的优化算法(比如Nelder-Mead方法)

Nelder-Mead

wikipedia
Nelder-Mead 算法

看起来像是一个解决超参数选择好办法, 后续有空会继续更新.