3.0 KiB
TensorFlow Serving
HOWTO Set up and run TensorFlow Serving. This is to this particular configuration.
Software
Main software in use:
- Debian
- Proxmox
- Ceph
- Python 3
- TensorFlow Serving
Installation
Install TensorFlow Serving. The recommended way is using docker
, but
here we build from source.
-
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md
-
https://github.com/tensorflow/serving/blob/master/tensorflow_serving/tools/docker/Dockerfile.devel
Install Bazel
Instead of building Bazel, download the version that TensorFlow Serving builds with,
currently version 1.2.1
as shown in the docker builder.
Install Bazel dependencies:
# Note, this installs python 2.7....
apt install g++ zlib1g-dev unzip python
Install bazel .deb from releases here.
Note get the version Serving wants, not the latest release.
wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb
wget https://github.com/bazelbuild/bazel/releases/download/1.2.1/bazel_1.2.1-linux-x86_64.deb.sha256
dpkg -i bazel_1.2.1-linux-x86_64.deb
apt -f install
Install Dependencies
Dependencies. Note, there are likely fewer dependencies than listed
in the docs, since bazel
is installed from .deb
, not built.
apt update
apt install --no-install-recommends \
automake \
build-essential \
ca-certificates \
curl \
git \
libcurl3-dev \
libfreetype6-dev \
libpng-dev \
libtool \
libzmq3-dev \
mlocate \
pkg-config \
python-dev \
software-properties-common \
swig \
unzip \
wget \
zip \
zlib1g-dev \
python3-distutils
Not installed:
openjdk-8-jdk \
openjdk-8-jre-headless \
Compile TensorFlow Serving
HOWTO compile TensorFlow Serving:
git clone https://github.com/tensorflow/serving
cd serving
git checkout 2.1.0
bazel build --color=yes --curses=yes \
${TF_SERVING_BAZEL_OPTIONS} \
--verbose_failures \
--output_filter=DONT_MATCH_ANYTHING \
${TF_SERVING_BUILD_OPTIONS} \
tensorflow_serving/model_servers:tensorflow_model_server
Build pip
package:
bazel build --color=yes --curses=yes \
${TF_SERVING_BAZEL_OPTIONS} \
--verbose_failures \
--output_filter=DONT_MATCH_ANYTHING \
${TF_SERVING_BUILD_OPTIONS} \
tensorflow_serving/tools/pip_package:build_pip_package
bazel-bin/tensorflow_serving/tools/pip_package/build_pip_package \
/tmp/pip
Install TensorFlow Server with pip
Install with pip
, don't use bazel, or build, etc.
# Set PATH, add to ~/.bashrc
export PATH="~/.local/bin:/usr/lib/ccache:$PATH"
. ~/.bashrc
sudo apt install python3-pip
pip3 install --upgrade --user pip
# Should return the one in ~/.local:
which pip3
# Install it, needs ~3 gigs free in /tmp
pip3 install --user tensorflow-serving-api
Misc
See also: