Exploring RedisAI
We have seen the optimization we can get through TorchScript, but what will we do with the optimized binary? Yes, we can load it in the C++ world and make a Go server, and load it there, but that's still painful.
Redis Labs and Orobix brought us another solution called RedisAI. It is a highly optimized runtime built on top of LibTorch and can accept compiled TorchScript binaries for serving through the Redis protocol. For people who don't have prior experience with Redis, http://redis.io has good documentation and the introduction doc given there [3] should be a good start.
RedisAI comes with three options to configure three backends: PyTorch, TensorFlow, and ONNX Runtime. It doesn't stop there: RedisAI uses DLPack in the ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access