cavis/nd4j/nd4j-remote/nd4j-json-server
Alex Black 0756e3fe70
Small fixes. (#206)
* Logging format tweaks for file logging

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Min abs error tweak for Util layer gradient checks

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* #8648 Fix SameDiff NPE instead of error for missing placeholders

Signed-off-by: AlexDBlack <blacka101@gmail.com>

* Test runtime reduction

Signed-off-by: AlexDBlack <blacka101@gmail.com>
2020-02-01 18:19:36 +11:00
..
src Small fixes. (#206) 2020-02-01 18:19:36 +11:00
README.md [WIP] Remote inference (#96) 2019-08-14 12:11:09 +03:00
pom.xml Unit/integration test split + test speedup (#166) 2020-01-22 22:27:01 +11:00

README.md

SameDiff model serving

This modules provides JSON-based serving of SameDiff models

Example

First of all we'll create server instance. Most probably you'll do it in application that will be running in container

val server = SameDiffJsonModelServer.<String, Sentiment>builder()
                .adapter(new StringToSentimentAdapter())
                .model(mySameDiffModel)
                .port(8080)
                .serializer(new SentimentSerializer())
                .deserializer(new StringDeserializer())
                .build();

server.start();
server.join();

Now, presumably in some other container, we'll set up remote inference client:

val client = JsonRemoteInference.<String, Sentiment>builder()
                .endpointAddress("http://youraddress:8080/v1/serving")
                .serializer(new StringSerializer())
                .deserializer(new SentimentDeserializer())
                .build();

Sentiment result = client.predict(myText);

On top of that, there's async call available, for cases when you need to chain multiple requests to one or multiple remote model servers.

Future<Sentiment> result = client.predictAsync(myText);