TensorFlow Serving

HOWTO Set up and run TensorFlow Serving. This is to this particular configuration.

Software

Main software in use:

Debian
Proxmox
Ceph
Python 3
TensorFlow Serving

Installation

Install TensorFlow Serving. The recommended way is using docker, but here we build from source.

Install Bazel

Instead of building Bazel, download the version that TensorFlow Serving builds with, currently version 1.2.1 as shown in the docker builder.

Install Bazel dependencies:

# Note, this installs python 2.7....
apt install g++ zlib1g-dev unzip python

Install bazel .deb from releases here.

firefox https://github.com/bazelbuild/bazel/releases

Note get the version Serving wants, not the latest release.

wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb
wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb.sha256
dpkg -i bazel_1.2.1-linux-x86_64.deb
apt -f install

Install Dependencies

Dependencies. Note, there are likely fewer dependencies than listed in the docs, since bazel is installed from .deb, not built.

apt update
apt install --no-install-recommends \
        automake \
        build-essential \
        ca-certificates \
        curl \
        git \
        libcurl3-dev \
        libfreetype6-dev \
        libpng-dev \
        libtool \
        libzmq3-dev \
        mlocate \
        pkg-config \
        python-dev \
        software-properties-common \
        swig \
        unzip \
        wget \
        zip \
        zlib1g-dev \
        python3-distutils

Not installed:

        openjdk-8-jdk \
        openjdk-8-jre-headless \

Compile TensorFlow Serving

HOWTO compile TensorFlow Serving:

git clone https://github.com/tensorflow/serving
cd serving

git checkout 2.1.0

bazel build --color=yes --curses=yes \
    ${TF_SERVING_BAZEL_OPTIONS} \
    --verbose_failures \
    --output_filter=DONT_MATCH_ANYTHING \
    ${TF_SERVING_BUILD_OPTIONS} \
    tensorflow_serving/model_servers:tensorflow_model_server

Build pip package:

bazel build --color=yes --curses=yes \
    ${TF_SERVING_BAZEL_OPTIONS} \
    --verbose_failures \
    --output_filter=DONT_MATCH_ANYTHING \
    ${TF_SERVING_BUILD_OPTIONS} \
    tensorflow_serving/tools/pip_package:build_pip_package


bazel-bin/tensorflow_serving/tools/pip_package/build_pip_package \
    /tmp/pip

Install TensorFlow Server with `pip`

Install with pip, don't use bazel, or build, etc.

# Set PATH, add to ~/.bashrc
export PATH="~/.local/bin:/usr/lib/ccache:$PATH"
. ~/.bashrc
sudo apt install python3-pip
pip3 install --upgrade --user pip
# Should return the one in ~/.local:
which pip3
# Install it, needs ~3 gigs free in /tmp
pip3 install --user tensorflow-serving-api

3.0 KiB Raw Permalink Blame History