Making your own model
This file has the methods for building and running nbox.Model.
Source file: Github
nbox.Model is a wrapper that detaches the underlying graph (pytorch, tensorflow, OpenVino, etc) and input object that dramatically improves the usability and ease of access of neural network models.
Initially we started with different classes for local-infer and cloud-infer, now they are both handled in the same file itself. This again might seem counter intuitive but since there are only two modes, adding if/else conditionals is okay!

Local-Infer Models

It has methods and functions that make the deployment and exporting super easy. Initialise each Model with underlying graph and category for each input item:
1
# initial model
2
model = Model(
3
my_clip_model(),
4
category = {
5
"input_images": "image",
6
"input_string": "text"
7
}
8
)
Copied!
Each Model looks performs the operations in the following order:
    1.
    Initialisation: Each Model gets loaded with underlying DAG and categories for the input
    2.
    Call: When the model gets an input the it is first sent to _handle_input_object() that takes in the user input and parses it based on parser for that particular input key.
In case of local models, the pre-post processing happens on your machine only.

Cloud-Infer Models

Once loaded the you can load the model by getting the URL and API key from the dashboard. We recommend that you load using nbox.load method but technically you can load Model directly as well.
Recommended
Second Method
1
model = nbox.load(
2
"https://api.nimblebox.ai/user/my_big_model/",
3
"nbxdeploy_zpO8I8AVzvOetQYAZanzP2mMgJ5oh84LG0wZdgh3U"
4
)
Copied!
1
model = nbox.Model(
2
"https://api.nimblebox.ai/user/my_big_model/",
3
"nbxdeploy_zpO8I8AVzvOetQYAZanzP2mMgJ5oh84LG0wZdgh3U"
4
)
Copied!
Currently the processing happens on the client side machine. We are building high performance Rust servers for offloading that task to cloud as well. That will significantly improve the latency and throughput of the service.---

Relevant Attributes

Each Model will have the following attributes that can be useful for debugging so watch out for these:
    1.
    __on_cloud: boolean that tells if this is loaded on cloud or not
    2.
    image_parser, text_parser: this is the image parser and text parser objects. The only difference between cloud version and local version is post_proc_fn, local converts to torch Tensors while cloud converts to lists.
    3.
    nbox_meta: not always initialised, but when present all the pre-post operations will happen according to this.

Documentation

class Model

__init__

1
def __init__(
2
model_or_model_url,
3
nbx_api_key: str = None,
4
category: str = None,
5
tokenizer=None,
6
model_key: str = None,
7
model_meta: dict = None,
8
verbose: bool = False,
9
)
Copied!
Model class designed for inference.
Arguments:
    model_or_model_url [torch.nn.Module, str] - either the model to be wrapped or just the URL from the deployed service.
    nbx_api_key str - API key for this deployed model, needs to be provided
    category (str, optional): Input categories for each input type to the model. Defaults to None.
    tokenizer ([transformers.PreTrainedTokenizer], optional): If this is a text model then tokenizer for this. Defaults to None.
    model_key (str, optional): With what key is this model initialised, useful for public models. Defaults to None.
    model_meta (dict, optional): Extra metadata when starting the model. Defaults to None.
    verbose (bool, optional): If true provides detailed prints. Defaults to False.
Raises:
    ValueError - If any category is "text" and tokenizer is not provided.

__call__

1
def __call__(input_object) -> Any
Copied!
This is the most important part of this codebase. The input_object can be anything from a tensor, an image file, file path as string, string to process as NLP model. This __call__ should understand the different use cases and manage accordingly.
The current idea is that what ever the input, based on the category (image, text, audio, smell) it will be parsed through dedicated parsers that can make ingest anything.
The entire purpose of this package is to make inference chill.
Arguments:
    input_object (Any) - input to be processed
    return_inputs (bool, optional): whether to return the inputs or not. Defaults to False.
Returns:
    Any - currently this is output from the model, so if it is tensors and return dicts.
Internally this call handles conditionals for cloud inference and local inference.

get_nbox_meta

1
def get_nbox_meta(
2
input_object
3
)
Copied!
nbox_meta is our way of storing the metadata for each model. This can contains two things:
    metadata: this is the trace and contains input and output for the given input and looks like this:
1
{'inputs': {'input_0': {'dtype': 'torch.float32',
2
'name': 'input_0',
3
'tensorShape': {'dim': [{'name': '', 'size': 1},
4
{'name': '', 'size': 3},
5
{'name': '', 'size': 244},
6
{'name': '', 'size': 244}],
7
'unknownRank': False}}},
8
'outputs': {'output_0': {'dtype': 'torch.float32',
9
'name': 'output_0',
10
'tensorShape': {'dim': [{'name': '', 'size': 1},
11
{'name': '', 'size': 1000}],
12
'unknownRank': False}}}}
Copied!
    spec: this has the details around model itself, and looks like this:
1
{
2
"category": "text",
3
"model_key": "transformers/distilroberta-base",
4
"name": "uniform_bumper_69ad"
5
}
Copied!

deploy

1
def deploy(
2
input_object: Any,
3
model_name: str = None,
4
cache_dir: str = None
5
)
Copied!
One-click-deploy your model on NBX platform.
Arguments:
    input_object Any - input to be processed
    model_name str, optional - custom model name for this model. Defaults to None.
    cache_dir str, optional - Custom caching directory. Defaults to None.
Returns:
    str, None - if deployment is successful then push then return the URL endpoint else return None
    folder_path str - folder where to put things in
Last modified 23d ago