1. ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
   安装tensorflow报ImportError: libcublas.so.9.0: cannot open shared object file的解决方法（2018年4月）

2. TensorFlow 识别不到GPU / TensorFlow GPU support
   通过代码

import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))

可以看到目前设备对TensorFlow的支持. 以如下为例, 系统目前没有识别出安装的1080Ti显卡, 因此只能看到CPU.

正常识别出显卡的效果如下:

没有识别出显卡的原因有很多, 最为常见的是 TensorFlow 版本和cuDNN, Cuda Toolkit的版本不匹配, 或是安装的时候没有将路径添加到系统路径中导致无法找到.
可以通过查询经过测试的构建配置 / tested_build_configurations来获取TensorFlow, cuDNN, Cuda Toolkit版本对应关系.

Translate from original page : tfx-serving-architeture

---

Architecture 架构

TensorFlow Serving is a flexible, high-performance serving system for machine learning models, designed for production environments. TensorFlow Serving makes it easy to deploy new algorithms and experiments, while keeping the same server architecture and APIs. TensorFlow Serving provides out of the box integration with TensorFlow models, but can be easily extended to serve other types of models.

TensorFlow Serving 是一个为了工业环境而设计的, 服务于机器学习模型的高效, 灵活的服务系统. TensorFlow Serving 能够在保持同样的服务架构和API的情况下让部署新的算法和实验更加容易. TensorFlow Serving 提供了和利用TensorFlow构建的模型非常契合的交互方式, 同时也可以很方便的扩展到其他类型的模型.

Key Concepts 关键概念

To understand the architecture of TensorFlow Serving, you need to understand the following key concepts:

未来理解 TensorFlow Serving的架构, 你需要理解以下关键概念:

Servables

Servables are the central abstraction in TensorFlow Serving. Servables are the underlying objects that clients use to perform computation (for example, a lookup or inference).

Servables 是 TensorFlow Serving 中的关键抽象概念. Servables 是一种底层的对象, 从而客户端可以利用它们进行计算(例如, 查询或者推断).

The size and granularity of a Servable is flexible. A single Servable might include anything from a single shard of a lookup table to a single model to a tuple of inference models. Servables can be of any type and interface, enabling flexibility and future improvements such as:

Servable 的大小和粒度都是非常灵活的. 单个 Servable 可以包含查询表的一个分块, 或者单个模型, 或者一组推断模型. Servables可以是任何类型或者接口, 未来的改进如下:

• streaming results
• experimental APIs
• asynchronous modes of operation

Servables do not manage their own lifecycle.

Servables 不管理它们自己的声明周期.

Typical servables include the following:

经典的 Servables 包含如下:

• a TensorFlow SavedModelBundle (tensorflow::Session)
• a lookup table for embedding or vocabulary lookups

Servable Versions

TensorFlow Serving can handle one or more versions of a servable over the lifetime of a single server instance. This enables fresh algorithm configurations, weights, and other data to be loaded over time. Versions enable more than one version of a servable to be loaded concurrently, supporting gradual rollout and experimentation. At serving time, clients may request either the latest version or a specific version id, for a particular model.

TensorFlow Serving 可以在单个服务器实例上处理一个Servable的一个或多个版本. 这意味着可以随着时间的变化, 用户可以刷新算法配置, 权重, 或者加载其他数据. 可同时加载一个 Servable 的多个版本, 支持逐步推广和实验. 在服务时间内, 客户端可以请求最新的版本, 或者根据版本id请求特定的版本.

Servable Streams

A servable stream is the sequence of versions of a servable, sorted by increasing version numbers.

Servable Stream 是 Servable 依照版本号进行升序排列的版本序列.

Models

TensorFlow Serving represents a model as one or more servables. A machine-learned model may include one or more algorithms (including learned weights) and lookup or embedding tables.

TensorFlow Serving 将 model 表示为一个或多个 Servable. 一个机器学习模型可能会包含一个或多个算法(包括学习权重), 查询表和嵌入表等.

You can represent a composite model as either of the following:

你可以将一个复杂模型通过如下的方式进行表示:

• multiple independent servables 多个独立 Servable
• single composite servable 单个复杂 Servable

A servable may also correspond to a fraction of a model. For example, a large lookup table could be sharded across many TensorFlow Serving instances.

一个 Servable 同样也可以相当于是一个模型的一部分. 举例来说, 一个大型的查询表可以被分割并表示为多个 TensorFlow Serving 实例.

Loaders

Loaders manage a servable’s life cycle. The Loader API enables common infrastructure independent from specific learning algorithms, data or product use-cases involved. Specifically, Loaders standardize the APIs for loading and unloading a servable.

Loaders 管理 Servable 的生命周期. Loader API 建立了脱离特定学习算法, 数据和产品用户案例的, 通用的结构. 需要特别提出的是, Loader 将加载和卸载 Servable 的API进行了标准化.

Sources

Sources are plugin modules that find and provide servables. Each Source provides zero or more servable streams. For each servable stream, a Source supplies one Loader instance for each version it makes available to be loaded. (A Source is actually chained together with zero or more SourceAdapters, and the last item in the chain emits the Loaders.)

Sources 是用来寻找和提供 Servable 的插件模块. 每个 Source 都提供了零或多个 Servable Stream.

TensorFlow Serving’s interface for Sources can discover servables from arbitrary storage systems. TensorFlow Serving includes common reference Source implementations. For example, Sources may access mechanisms such as RPC and can poll a file system.

TensorFlow Serving 中 Source 的接口可以从任意存储系统中发现 Servable. TensorFlow Serving 包含了通用的 Source 实现. 例如, Source 可以访问类似RPC的机制并挂载一个文件系统.

Sources can maintain state that is shared across multiple servables or versions. This is useful for servables that use delta (diff) updates between versions.

Sources 可以在多个 Servable 或版本中维持状态. 这对于使用版本件不同进行更新的 Servable 非常有用.

Aspired Versions

Aspired versions represent the set of servable versions that should be loaded and ready. Sources communicate this set of servable versions for a single servable stream at a time. When a Source gives a new list of aspired versions to the Manager, it supercedes the previous list for that servable stream. The Manager unloads any previously loaded versions that no longer appear in the list.

Aspired Versions 表示的是可以被加载和读取的 Servable 版本的集合. Source 每次访问这个集合来来获取单个 Servable Stream. 当 Source 交给 Manager 一个新的关于 Aspired Version 的列表时, 它将取代之前的列表. Manager 卸载任何之前存在但现在不存在的版本.

See the advanced tutorial to see how version loading works in practice.

查看advanced tutorial 来获取关于实际应用中版本加载如何工作的详情.

Managers

Managers handle the full lifecycle of Servables, including:

Manager 负责处理 Servable 的全部生命周期, 例如

• loading Servables
• serving Servables
• unloading Servables

Managers listen to Sources and track all versions. The Manager tries to fulfill Sources’ requests, but may refuse to load an aspired version if, say, required resources aren’t available. Managers may also postpone an “unload”. For example, a Manager may wait to unload until a newer version finishes loading, based on a policy to guarantee that at least one version is loaded at all times.

Manger 监听 Source 并追踪所有的版本. Manager 尝试满足 Source 的请求, 但可能会拒绝加载一个条件不允许的 Aspired Version, 例如, 资源不存在. Manager 也可能会延迟卸载的操作. 例如, 一个基于任何时刻都必须保持至少有一个版本可用侧率的 Manager 可能会等到新的版本完成加载后再进行老版本的卸载.

TensorFlow Serving Managers provide a simple, narrow interface — GetServableHandle() — for clients to access loaded servable instances.

TensorFLow Serving Manager 提供了一个简单, 限定的接口 GetServableHandle() 让客户端去访问加载的 Servable 实例.

Core

Using the standard TensorFlow Serving APis, TensorFlow Serving Core manages the following aspects of servables:

使用标准的 TensroFlow Serving API, TensorFlow Serving Core 管理 Servable的以下方面:

• lifecycle 生命周期
• metrics 指标

TensorFlow Serving Core treats servables and loaders as opaque objects.

TensorFlow Serving Core 将 Servable 和 Loader 当做黑盒进行处理.

Life of a Servable

Broadly speaking:

大概来说:

1. Sources create Loaders for Servable Versions.
2. Source 创建 Loader 来获取 Servable Version.

3. Loaders are sent as Aspired Versions to the Manager, which loads and serves them to client requests.

4. Loader 被当做 Aspired Version 发送给 Manager, Manager 加载并使用 Loader 来处理客户端请求.

In more detail:

细节上来说:

1. A Source plugin creates a Loader for a specific version. The Loader contains whatever metadata it needs to load the Servable.
2. Source 插件为一个 Loader 创建一个指定的版本. Loader 包含了所有加载 Servable 所需的元数据.

3. The Source uses a callback to notify the Manager of the Aspired Version.

4. Source 通过回调函数通知 Manager Aspired Version的相关信息.

5. The Manager applies the configured Version Policy to determine the next action to take, which could be to unload a previously loaded version or to load the new version.

6. Manager 依照配置的版本策略决定该如何进行下一步操作,可能是卸载老版本, 或者加载新版本.

7. If the Manager determines that it’s safe, it gives the Loader the required resources and tells the Loader to load the new version.

8. 如果 Manager 判断当前环境安全, 则 Manager 赋予 Loader 必要的资源并通知 Loader 可以加载新版本.

9. Clients ask the Manager for the Servable, either specifying a version explicitly or just requesting the latest version. The Manager returns a handle for the Servable.

10. 客户端向 Manger 请求 Servable, 不管是指定版本或最新版本. Manager 返回该 Servable 的操作接口.

For example, say a Source represents a TensorFlow graph with frequently updated model weights. The weights are stored in a file on disk.

举例来说, 假设有一个 Source, 其表示一张经常更新模型权重的 TensorFlow 图. 权重数据存储在硬盘上的某一个文件里.

1. The Source detects a new version of the model weights. It creates a Loader that contains a pointer to the model data on disk.
2. Source 检测到模型权重的新版本后, 创建一个包含指向硬盘上模型数据的指针的 Loader.

3. The Source notifies the Dynamic Manager of the Aspired Version.

4. Source 通知 Dynamic Manager 相关 Aspired Version 信息.

5. The Dynamic Manager applies the Version Policy and decides to load the new version.

6. Dynamic Manager 依照版本策略决定是否/何时加载新版本.

7. The Dynamic Manager tells the Loader that there is enough memory. The Loader instantiates the TensorFlow graph with the new weights.

8. Dynamic 通知 Loader, 内存可用. Loader 利用新的权重数据进行 TensorFlow Graph的实例化.

9. A client requests a handle to the latest version of the model, and the Dynamic Manager returns a handle to the new version of the Servable.

10. 客户端请求模型最新版本的操作接口. Dynamic Manager 返回最新版本的操作接口.

Extensibility 扩展性

TensorFlow Serving provides several extension points where you can add new functionality.

TensorFlow Serving 提供了多个可扩展点, 方便用户添加新功能.

Version Policy 版本管理

Version Policies specify the sequence of version loading and unloading within a single servable stream.

版本管理明确了在单个 Servable Stream 中加载和卸载版本的顺序.

TensorFlow Serving includes two policies that accommodate most known use- cases. These are the Availability Preserving Policy (avoid leaving zero versions loaded; typically load a new version before unloading an old one), and the Resource Preserving Policy (avoid having two versions loaded simultaneously, thus requiring double the resources; unload an old version before loading a new one). For simple usage of TensorFlow Serving where the serving availability of a model is important and the resource costs low, the Availability Preserving Policy will ensure that the new version is loaded and ready before unloading the old one. For sophisticated usage of TensorFlow Serving, for example managing versions across multiple server instances, the Resource Preserving Policy requires the least resources (no extra buffer for loading new versions).

TensorFlow Serving 对于大部分的用户案例提供了两个版本管理策略, 分别是 Availability Preserving Policy (避免没有版本加载的情况; 通常先加载新版本后卸载旧版本), 和 Resource Preserving Policy (避免同时拥有两个可用版本, 也就是资源的双倍占用; 在加载新版本之前卸载就版本). 在模型的服务可用性较为重要, 资源消耗低的简单环境下, Availability Preserving Policy 可以保证新版本在老版本卸载之前被加载. 在复杂的环境下, 例如, 多个服务实例上管理多个版本, Resource Preserving Policy 需要的资源会更少(没有加载新版本的额外资源需求)

Source

New Sources could support new filesystems, cloud offerings and algorithm backends. TensorFlow Serving provides some common building blocks to make it easy & fast to create new sources. For example, TensorFlow Serving includes a utility to wrap polling behavior around a simple source. Sources are closely related to Loaders for specific algorithms and data hosting servables.

新的 Source 可以支持新的文件系统, 云服务和算法后端. TensorFlow Serving 提供了一些通用的 building block 用来更简单更快的创建 Source. 举例来说, TensorFlow Serving 提供了装饰挂载在一个简单 Source 的方法. Source 因为特定的算法和数据支持 Servable 和 Loader 关联密切.

See the Custom Source document for more about how to create a custom Source.

查看Custom Source获取更多自定义 Source 的详情.

Loaders

Loaders are the extension point for adding algorithm and data backends. TensorFlow is one such algorithm backend. For example, you would implement a new Loader in order to load, provide access to, and unload an instance of a new type of servable machine learning model. We anticipate creating Loaders for lookup tables and additional algorithms.

Loader 是用来添加算法和数据后端的扩展点. TensorFlow 就是这样的一个算法后端. 例如, 如果你为了加载,提供访问和卸载一个新的机器学习模型而实现一个新的 Loader. 我们预先为查询表和增加的算法创建 Loader.

See the Custom Servable document to learn how to create a custom servable.

查看Custom Servable获取更多如何自定义 Servable 的详情.

Batcher

Batching of multiple requests into a single request can significantly reduce the cost of performing inference, especially in the presence of hardware accelerators such as GPUs. TensorFlow Serving includes a request batching widget that lets clients easily batch their type-specific inferences across requests into batch requests that algorithm systems can more efficiently process. See the Batching Guide for more information.

在单次请求中汇聚多个请求形成 Batching 可以有效降低模型推断结果的资源消耗, 特别是在例如GPU之类的加速器面前. TensorFlow Serving 包含了一个批量请求装饰器, 可以让客户端更容易的将他们的独特类型的请求放在一起, 从而让算法系统能够更有效的处理. 查看Batching Guide获取更多详情.

DataFrame 根据某一列值进行排序

df.sort_values(“xxx”,inplace=True)

DataFrame 根据多列值进行排序

DataFrame 筛选某一列值中的前百分之X的记录

临时解决办法:
假设DataFrame为DFI(DataFrameInstance), 目标列为A, 筛选列A中值前25%的记录, 则先计算 goal_value = DFI[‘A’].quantile(.75) 计算出在75%位置的数值是多少, 然后通过 DFI[DFI[‘A’] > goal_value] 来达到效果.

遍历 DataFrame

DataFrame.iterrows() 返回的结果是index和row的tuple. 其中row是Series类型.

在 Python深度学习 一书中7.33节中提高到了
为了找到一组更好的集成权重, 你可以使用随机搜索或简单的优化算法(比如Nelder-Mead方法)

Nelder-Mead

wikipedia
Nelder-Mead 算法

看起来像是一个解决超参数选择好办法, 后续有空会继续更新.

docker: Error response from daemon: Get https://registry-1.docker.io/v2/tensorflow/serving/manifests/nightly-devel: unauthorized: incorrect username or password.

可以通过Docker –> Preferences –> Daemon，在Registry mirrors中添加一条：https://registry.docker-cn.com，
重启Docker，重新pull即可解决问题。Docker使用国内镜像加速可参考Docker 中国官方镜像加速

Docker拉取镜像报错unauthorized: incorrect username or password的解决方法

MovieLens

MovieLens 是一个常见的数据集合, 经常用来做推荐效果的测试.
Tensorflow 提供了基于 Wide&Deep 模型的一个基础模型. 是用Estimator进行实现的. 方便进行分布式的训练.

项目文件

Tensorflow的官方模型中, 在 Wide & Deep 这一模型下分别实现了 Census 和 MovieLens 两个数据集上的实现. 这里我们只介绍 MovieLens.
相关的文件如下所示 ( 2019-03-20 : 把movies.py 单独放在datasets文件夹下有些奇怪 ) :

model/oficial/wide_deep/

• movielens_dataset.py
• movielens_main.py
• movielens_test.py
• wide_deep_run_loop.py

model/official/datasets/

• movielens.py

学习到的小技巧

1. 将 CSV 文件生成 DataFrame 写入 Buffer 中, 方便 DataSet 读取.
2. 将DataFrame中的某一列转换为向量.

将 CSV 文件生成 DataFrame 写入 Buffer 中, 方便 DataSet 读取

将DataFrame中的某一列转换为向量

"""
__file_name__ : movielens.py
"""
def integerize_genres(dataframe):
  """Replace genre string with a binary vector.
  Args:
    dataframe: a pandas dataframe of movie data.
  Returns:
    The transformed dataframe.
  """
  def _map_fn(entry):
    entry.replace("Children's", "Children")  # naming difference.
    movie_genres = entry.split("|")
    output = np.zeros((len(GENRES),), dtype=np.int64)
    for i, genre in enumerate(GENRES):
      if genre in movie_genres:
        output[i] = 1
    return output

  dataframe[GENRE_COLUMN] = dataframe[GENRE_COLUMN].apply(_map_fn)

  return dataframe

tf.feature_column api_doc

categorical_column_with_hash_bucket(…): Represents sparse feature where ids are set by hashing.

categorical_column_with_identity(…): A CategoricalColumn that returns identity values.

categorical_column_with_vocabulary_file(…): A CategoricalColumn with a vocabulary file.

categorical_column_with_vocabulary_list(…): A CategoricalColumn with in-memory vocabulary.

categorical_column_with_identity

categorical_column_with_hash_bucket

categorical_column_with_vocabulary_file

A CategoricalColumn with a vocabulary file.

Use this when your inputs are in string or integer format, and you have a vocabulary file that maps each value to an integer ID. By default, out-of-vocabulary values are ignored. Use either (but not both) of num_oov_buckets and default_value to specify how to include out-of-vocabulary values.

For input dictionary features, features[key] is either Tensor or SparseTensor. If Tensor, missing values can be represented by -1 for int and '' for string, which will be dropped by this feature column.

Example with num_oov_buckets:
File ‘/us/states.txt’ contains 50 lines, each with a 2-character U.S. state abbreviation. All inputs with values in that file are assigned an ID 0-49, corresponding to its line number. All other values are hashed and assigned an ID 50-54.

states = categorical_column_with_vocabulary_file(
    key='states', vocabulary_file='/us/states.txt', vocabulary_size=50,
    num_oov_buckets=5)
columns = [states, ...]
features = tf.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction = linear_model(features, columns)

Example with default_value:
File ‘/us/states.txt’ contains 51 lines – the first line is ‘XX’, and the other 50 each have a 2-character U.S. state abbreviation. Both a literal ‘XX’ in input, and other values missing from the file, will be assigned ID 0. All others are assigned the corresponding line number 1-50.

states = categorical_column_with_vocabulary_file(
    key='states', vocabulary_file='/us/states.txt', vocabulary_size=51,
    default_value=0)
columns = [states, ...]
features = tf.parse_example(..., features=make_parse_example_spec(columns))
linear_prediction, _, _ = linear_model(features, columns)

And to make an embedding with either:

columns = [embedding_column(states, 3),...]
features = tf.parse_example(..., features=make_parse_example_spec(columns))
dense_tensor = input_layer(features, columns)

Args:

key: A unique string identifying the input feature. It is used as the
column name and the dictionary key for feature parsing configs, feature Tensor objects, and feature columns.
vocabulary_file: The vocabulary file name.
vocabulary_size: Number of the elements in the vocabulary. This must be no greater than length of vocabulary_file, if less than length, later values are ignored. If None, it is set to the length of vocabulary_file.
dtype: The type of features. Only string and integer types are supported.
default_value: The integer ID value to return for out-of-vocabulary feature values, defaults to -1. This can not be specified with a positive num_oov_buckets. num_oov_buckets: Non-negative integer, the number of out-of-vocabulary buckets. All out-of-vocabulary inputs will be assigned IDs in the range [vocabulary_size, vocabulary_size+num_oov_buckets) based on a hash of the input value. A positive num_oov_buckets can not be specified with default_value.

Returns:
A CategoricalColumn with a vocabulary file.

Raises:
ValueError: vocabulary_file is missing or cannot be opened.
ValueError: vocabulary_size is missing or < 1.
ValueError: num_oov_buckets is a negative integer.
ValueError: num_oov_buckets and default_value are both specified.
ValueError: dtype is neither string nor integer.

categorical_column_with_vocabulary_list

根据方法的介绍可以看出, default_value 为默认oov(out-of-vocabulary)单词的对应值.

作为新手, 你可能还在使用feed_dict来给你的tensorflow 图提供数据. 但是, Tensorflow已经在tf.data中提供了更高层的API, 用于提供数据.
As green hand, you can use feed_dict to feed data to your tensorflow graph. However, Tensorflow has provided high-level API for feeding data in tf.data.

Basic Concept

tf.data.Dataset 用来表征数据.
tf.data.Iterator 用来消费数据.

Dataset.make_one_shot_iterator() 不需要显性的初始化.

由Dataset创建的Iterator需要执行initializer().
而Iterator实例化的对象需要通过make_initializer()来指定迭代的Dataset.

根据Iterator的定义,
Acccording to the definition of Iterator,

注意 : 大部分用户并不会直接调用, 而是用Dataset.make_initializable_iterator() or Dataset.make_one_shot_iterator()来间接生成.

Note: Most users will not call this initializer directly, and will instead use Dataset.make_initializable_iterator() or Dataset.make_one_shot_iterator().

但是, 如果你知道dataset的数据类型和结构, 可以通过调用staticmethod方法 from_structure 来实例化一个Iterator.

However, if you know the specific type and structure of dataset, you can generate one Iterator instance by calling the static method from_structure

---

Differences between from_tensors and from_tensor_slices

它的作用是切分传入Tensor的第一个维度，生成相应的dataset。
例1：
dataset = tf.data.Dataset.from_tensor_slices(np.random.uniform(size=(5, 2)))
传入的数值是一个矩阵，它的形状为(5, 2)，tf.data.Dataset.from_tensor_slices就会切分它形状上的第一个维度，最后生成的dataset中一个含有5个元素，每个元素的形状是(2, )，即每个元素是矩阵的一行。
例2：
对于更复杂的情形，比如元素是一个python中的元组或者字典：在图像识别中一个元素可以是｛”image”:image_tensor,”label”:label_tensor｝的形式。
dataset = tf.data.Dataset.from_tensor_slices ( { “a”:np.array([1.0,2.0,3.0,4.0,5.0]), “b”:np.random.uniform(size=(5,2) ) } )
这时，函数会分别切分”a”中的数值以及”b”中的数值，最后总dataset中的一个元素就是类似于{ “a”:1.0, “b”:[0.9,0.1] }的形式。
[CSDN ][https://blog.csdn.net/qi_1221/article/details/79460875]

Dataset.from_tensors()
Dataset.from_tensor_slices()

1. Find some file with unnecessary warning ‘Permission denied’ ?
   A : find [path] -name "pattern" 2>/dev/null to brush those annoying waring off.