Introduction to our SDK

API documentation for nbox package.

NimbleBox.ai started to reduce entry barrier to production grade AI workloads for developers. We started that by introducing our platform, and now with our SDK nbox. Inference of models is the most common operation we see on our platform, and is a bottleneck for non-ML developers who just want to use that as a function and don't want to setup a pipeline.

Find the complete open source code on Github. Install the package by running the command:

pip install nbox

In order to effectively use this package, you must have a password set. You can get it by going to Profile β†’ Reset Password.

When loading nbox for the first time, it will prompt you the username and password and create a secrets file at ~/.nbx/secrets.json. This file then contains all the information that you don't have to fetch manually again.

nbox is a new package designed ground up with inference and production grade deployment in mind. The input to model is called input_object and it can be a string, array-like, binary-like in form of a list or Dict. Read full technical document here.

import nbox
​
# As all these models come from the popular frameworks you use such as
# torchvision, efficient_pytorch or hf.transformers
model = nbox.load("torchvision/mobilenetv2", pretrained = True)
​
# nbox makes inference the priority so you can
out = model('cat.jpg') # pass it image path
out = model(np.array(Image.open('cat.jpg'))) # pass it numpy arrays
out = model(['cat.jpg', Image.open('cat.jpg')]) # pass it a list for batch inference

nbox can load any underlying model from package and can consume anything (eg. code above) whether it is filepaths, PIL.Image or just a numpy array.

Deployment

This package can always be used as a standalone package however it also integrates tightly with NimbleBox.ai's deployment services. This is how easy it is to deploy your model using nbox. Read more about deployment in API documentation.

# load any model and simply provide your API key and machine_id
model = nbox.load("transformers/gpt2", pretrained = True)
url_endpoint, api_key = model.deploy((
"This is some sample input string, it can be "
"numpy array or list of strings. You don't need "
"to take the headache of conversion to strings."
))

Once deployment starts see your terminal light up!

Screenshot from our platform

Cloud Infer

Often your local machine does not have enough power to run large models that you need for your task. Instead use cloud infer for this and feel no difference in experience, but in the background you will make an API call to NimbleBox.ai managed deployments. When loading the model simply pass the URL and keys from your dashboard.

Copy the URL and API key from the dashboard and simply pass that to nbox.load.

[WIP] We have support for most common pre-post operations from inside nbox however python is too slow for production services and so we are building Rust intermediate layers that will handle intermediate workloads using multicore code, giving very low latencies.

# load by passing the URL and nbx_api_key
model = nbox.load(
"https://api.nimblebox.ai/user/my_big_model/",
"nbxdeploy_zpO8I8AVzvOetQYAZanzP2mMgJ5oh84LG0wZdgh3U"
)
​
# use as if it runs locally
pred = model("https://s3.aws.com/bucket/my_info.jpg")
out = model('cat.jpg') # pass it image path
out = model(np.array(Image.open('cat.jpg'))) # pass it numpy arrays
out = model(['cat.jpg', Image.open('cat.jpg')]) # pass it a list for batch inference

Extending nbox

While nbox has all the things required for image and text based models, you can add your own custom pipelines by writing just one function. Read more in documentation:

  1. How to add your own parsers? link​

  2. Deploying your own model link​

  3. Loading your own model link​

  4. Raise Feature Requests link​

Documentation Index

  1. ​Parsers​

    1. ​Image Parser​

    2. ​Text Parser​

    3. ​Add your parser​

  2. ​Loading​

  3. ​Model​

Hope you enjoy this and it makes your life easier!