Endpoints in our platform provide a mechanism for creating services that accept predictions via a designated endpoint. These services, based on existing platform versions, leverage Cairo under the hood to ensure provable inferences. Using the CLI, users can effortlessly deploy and retrieve information about these machine learning services.
Deploying a model as an endpoint
To deploy a model, you must first have a version of that model. This can be done by creating a version of your model.
To create a new service, users can employ the deploy command. This command facilitates the deployment of a verifiable machine learning service ready to accept predictions at the /cairo_run endpoint, providing a straightforward method for deploying and utilizing machine learning capabilities.
> giza endpoints deploy --model-id 1 --version-id 1 model.sierra.json▰▰▰▰▰▱▱ Creating endpoint![giza][2024-02-07 12:31:02.498] Endpoint is successful ✅[giza][2024-02-07 12:31:02.501] Endpoint created with id -> 1 ✅[giza][2024-02-07 12:31:02.502] Endpoint created with endpoint URL: https://deployment-gizabrain-38-1-53427f44-dagsgas-ew.a.run.app 🎉
If a model is fully compatible the sierra file is not needed and can be deployed without using it in the command:
> giza endpoints deploy --model-id 1 --version-id 1
▰▰▰▰▰▱▱ Creating endpoint!
[giza][2024-02-07 12:31:02.498] Endpoint is successful ✅
[giza][2024-02-07 12:31:02.501] Endpoint created with id -> 1 ✅
[giza][2024-02-07 12:31:02.502] Endpoint created with endpoint URL: https://deployment-gizabrain-38-1-53427f44-dagsgas-ew.a.run.app 🎉
For a partially compatible model the sierra file must be provided, if not an error will be shown.
Example request
Now our service is ready to accept predictions at the provided endpoint URL. To test this, we can use the curl command to send a POST request to the endpoint with a sample input.
The list command is designed to retrieve information about all existing endpoints. It provides an overview of the deployed machine learning services, allowing users to monitor and manage multiple endpoints efficiently.
Executing this command will display a list of all current endpoints, including relevant details such as service names, version numbers, and endpoint status.
To list only active endpoints you can use the flag --only-active/-a so only active ones are shown.
Retrieving an endpoint
For retrieving detailed information about a specific endpoint, users can utilize the get command. This command allows users to query and view specific details of a single endpoint, providing insights into the configuration, status, and other pertinent information.
> giza endpoints get --endpoint-id 1>>>>{"id":1,"status":"COMPLETED","uri":"https://deployment-gizabrain-38-1-53427f44-dagsgas-ew.a.run.app","size":"S","service_name":"deployment-gizabrain-38-1-53427f44","model_id":38,"version_id":1,"is_active":true}
Retrieve logs for an endpoint
If an error has occurred or you want to know more about what is happening under the hood, you can retrieve the logs of the deployed endpoint.
> giza endpoints logs --endpoint-id 1
>>>>
[giza][2024-05-19 19:29:43.601] Getting logs for endpoint 1 ✅
2024-05-19T10:12:12.635831Z INFO orion_runner: ✅ Sierra program downloaded successfully!
2024-05-19T10:12:12.635905Z INFO orion_runner: 🚀 Server running on 0.0.0.0:8080
Delete an endpoint
For deleting an endpoint, users can use the delete command. This command facilitates the removal of a machine learning service, allowing users to manage and maintain their deployed services efficiently.
The endpoints are not fully deleted, so you can still access the underlying proofs generated by them.
List the proving jobs for an endpoint
To list the proving jobs for an endpoint, we can use the list-jobs command available for the endpoints. This command will return a list of all the proving jobs for the endpoint with the request_id for easier tracking.
To list the proofs for an endpoint, we can use the list-proofs command available for the endpoints. This command will return a list of all the proofs for the endpoint with the request_id for easier tracking.
After successfully creating a proof for your Orion Cairo model, the next step is to verify its validity. Giza offers a verification method using the verify command alongside the endpoint-id and proof-id.
