Create an AI Agent to Mint an MNIST NFT

Agents are entities designed to assist users in interacting with Smart Contracts by managing the proof verification of verifiable ML models and executing these contracts using Ape's framework.

Agents serve as intermediaries between users and Smart Contracts, facilitating seamless interaction with verifiable ML models and executing associated contracts. They handle the verification of proofs, ensuring the integrity and authenticity of data used in contract execution.

In this tutorial, we will create an AI Agent to mint an MNIST NFT. Using an existing MNIST endpoint, we will perfom a prediction on the MNIST dataset and mint an NFT based on the prediction.

This tutorial can be thought as the continuation of the previous tutorial Verifiable MNIST Neural Network.

Installation

To follow this tutorial, you must first proceed with the following installation.

Handling Python versions with Pyenv

You should install Giza tools in a virtual environment. If you’re unfamiliar with Python virtual environments, take a look at this guide. A virtual environment makes it easier to manage different projects and avoid compatibility issues between dependencies.

Install Python 3.11 using pyenv

pyenv install 3.11.0

Set Python 3.11 as local Python version:

pyenv local 3.11.0

Create a virtual environment using Python 3.11:

pyenv virtualenv 3.11.0 my-env

Activate the virtual environment:

pyenv activate my-env

Now, your terminal session will use Python 3.11 for this project.

Install Giza

Install Giza CLI

Install the CLI from PyPi

pip install giza-cli

Install Agent SDK

Install the Agents package from PyPi

pip install giza-agents

You'll find more options for installing Giza in the installation guide.

Install Dependencies

You must also install the following dependencies:

pip install torch scipy numpy

Setup

From your terminal, create a Giza user through our CLI in order to access the Giza Platform:

giza users create

After creating your user, log into Giza:

giza users login

Create an API Key for your user in order to not regenerate your access token every few hours.

giza users create-api-key

Before you begin

We assume that you already have an MNIST inference endpoint deployed on Giza. If not, please follow the Verifiable MNIST Neural Network tutorial to deploy an MNIST endpoint.

Creating an Ape Account (Wallet)

Before we can create an AI Agent, we need to create an account using Ape's framework. We can do this by running the following command:

$ ape accounts generate <account name>
Enhance the security of your account by adding additional random input:
Show mnemonic? [Y/n]: n
Create Passphrase to encrypt account:
Repeat for confirmation:
SUCCESS: A new account '0x766867bB2E3E1A6E6245F4930b47E9aF54cEba0C' with HDPath m/44'/60'/0'/0/0 has been added with the id '<account name>'

This will create a new account under $HOME/.ape/accounts using the keyfile structure from the eth-keyfile library , for more information on the account management, you can refer to the Ape's framework documentation.

In this account generation we will be prompted to enter a passphrase to encrypt the account, this passphrase will be used to unlock the account when needed, so make sure to keep it safe.

We encourage the creation of a new account for each agent, as it will allow you to manage the agent's permissions and access control more effectively, but importing accounts is also possible.

Funding the account

Before we can create an AI Agent, we need to fund the account with some ETH. You can do this by sending some ETH to the account address generated in the previous step.

In this case we will use Sepolia testnet, you can get some testnet ETH from a faucet like Alchemy Sepolia Faucet or LearnWeb3 Faucet. This faucets will ask for security meassures to make sure that you are not a bot like having a specific amount of ETH in mainnet or a Github account. There are many faucets available, you can choose the one that suits you the best.

Once, we can recieve the testnet ETH and we have a funded account, we can proceed to create an AI Agent.

Creating an AI Agent

Now that we have a funded account, we can create an AI Agent. We can do this by running the following command:

giza agents create --model-id <model-id> --version-id <version-id> --name <agent name> --description <agent description>

# or if you have the endpoint-id

giza agents create --endpoint-id <endpoint-id> --name <agent name> --description <agent description>

This command will prompt you to choose the account you want to use with the agent, once you select the account, the agent will be created and you will receive the agent id. The output will look like this:

[giza][2024-04-10 11:50:24.005] Creating agent ✅
[giza][2024-04-10 11:50:24.006] Using endpoint id to create agent, retrieving model id and version id
[giza][2024-04-10 11:50:53.480] Select an existing account to create the agent.
[giza][2024-04-10 11:50:53.480] Available accounts are:
┏━━━━━━━━━━━━━┓
┃  Accounts   ┃
┡━━━━━━━━━━━━━┩
│ my_account  │
└─────────────┘
Enter the account name: my_account
{
  "id": 1,
  "name": <agent_name>,
  "description": <agent_description>,
  "parameters": {
    "model_id": <model_id>,
    "version_id": <version_id>,
    "endpoint_id": <endpoint_id>,
    "alias": "my_account"
  },
  "created_date": "2024-04-10T09:51:04.226448",
  "last_update": "2024-04-10T09:51:04.226448"
}

This will create an AI Agent that can be used to interact with the deployed MNIST model.

How to use the AI Agent

Now that the agent is created we can start using it through the Agent SDK. As we will be using the agent to mint a MNIST NFT, we will need to provide the MNIST image to the agent and preprocess it before sending it to the model.

import pprint
import numpy as np
from PIL import Image

from giza.agents import GizaAgent, AgentResult

# Make sure to fill these in
MODEL_ID = <model-id>
VERSION_ID = <version-id>
# As we are executing in sepolia, we need to specify the chain
CHAIN = "ethereum:sepolia:geth"
# The address of the deployed contract
MNIST_CONTRACT = "0x7FE10f158b57CF9e48Af672EC7A43D0c4952da17"

Now we will start creating the functions that are necessary to create a program that will:

• Load and preprocess the MNIST image

• Create an instance of an agent

• Predict the MNIST image using the agent

• Access the prediction result

• Mint an NFT based on the prediction result

To load and preprocess the image we can use the function that we already created in the previous tutorial Verifiable MNIST Neural Network.

# This function is for the previous MNIST tutorial
def preprocess_image(image_path: str):
    """
    Preprocess an image for the MNIST model.

    Args:
        image_path (str): Path to the image file.

    Returns:
        np.ndarray: Preprocessed image.
    """

    # Load image, convert to grayscale, resize and normalize
    image = Image.open(image_path).convert('L')
    # Resize to match the input size of the model
    image = image.resize((14, 14))
    image = np.array(image).astype('float32') / 255
    image = image.reshape(1, 196)  # Reshape to (1, 196) for model input
    return image

Now, we will create a function to create an instance of the agent. The agent is an extension of the GizaModel class, so we can execute the predict as if we were using a model, but the agents needs more information:

• chain: The chain where the contract and account are deployed

• contracts: This is a dictionary in the form of {"contract_alias": "contract_address"} that contains the contract alias and address.

This contract_alias is the alias that we will use when executing the contract through code.

def create_agent(model_id: int, version_id: int, chain: str, contract: str):
    """
    Create a Giza agent for the MNIST model with MNIST 
    """
    agent = GizaAgent(
        contracts={"mnist": contract},
        id=model_id,
        version_id=version_id,
        chain=chain,
        account="my_account1"
    )
    return agent

This new task will execute the predict method of the agent following the same format as a GizaModel instance. But in this case, the agent will return an AgentResult object that contains the prediction result, request id and multiple utilities to handle the verification of the prediction.

def predict(agent: GizaAgent, image: np.ndarray):
    """
    Predict the digit in an image.

    Args:
        image (np.ndarray): Image to predict.

    Returns:
        int: Predicted digit.
    """
    prediction = agent.predict(
        input_feed={"image": image}, verifiable=True
    )
    return prediction

Once we have the result, we need to access it. The AgentResult object contains the value attribute that contains the prediction result. In this case, the prediction result is a number from 0 to 9 that represents the digit that the model predicted.

When we access the value, the execution will be blocked until the proof of the prediction has been created and once created it is verified. If the proof is not valid, the execution will raise an exception.

This task is more for didactic purposes, to showcase in the workspaces the time that it can take to verify the proof. In a real scenario, you can use the AgentResult value directly in any other task.

def get_digit(prediction: AgentResult):
    """
    Get the digit from the prediction.

    Args:
        prediction (dict): Prediction from the model.

    Returns:
        int: Predicted digit.
    """
    # This will block the executon until the prediction has generated the proof and the proof has been verified
    return int(prediction.value[0].argmax())

Finally, we will create a task to mint an NFT based on the prediction result. This task will use the prediction result to mint an NFT with the image of the MNIST digit predicted.

To execute the contract we have the GizaAgent.execute context that will yield all the initialized contracts and the agent instance. This context will be used to execute the contract as it handles all the connections to the nodes, networks, and contracts.

In our instantiation of the agent we added an mnist alias to access the contract to ease its use. For example:

# A context is created that yields all the contracts used in the agent
with agent.execute() as contracts:
    # This is how we access the contract
    contracs.mnist
    # This is how we access the functions of the contract
    contracts.mnsit.mint(...)

def execute_contract(agent: GizaAgent, digit: int):
    """
    Execute the MNIST contract with the predicted digit to mint a new NFT.

    Args:
        agent (GizaAgent): Giza agent.
        digit (int): Predicted digit.

    Returns:
        str: Transaction hash.
    """
    with agent.execute() as contracts:
        contract_result = contracts.mnist.mint(digit)
    return contract_result

Now that we have all the steps defined, we can create the function to execute.

def mint_nft_with_prediction():
    # Preprocess image
    image = preprocess_image("seven.png")
    # Create Giza agent
    agent = create_agent(MODEL_ID, VERSION_ID, CHAIN, MNIST_CONTRACT)
    # Predict digit
    prediction = predict(agent, image)
    # Get digit
    digit = get_digit(prediction)
    # Execute contract
    result = execute_contract(agent, digit)
    pprint.pprint(result)

Remember that we should have kept the passphrase in a safe place? Now it is time to use it. For learning purposes, we will use the passphrase in the code, but in a real scenario, you should keep it safe and not hardcode it in the code.

# DO NOT COMMIT YOUR PASSPHRASE
import os
os.environ['MY_ACCOUNT1_PASSPHRASE'] = 'a'

Now let's execute mint_nft_with_prediction.

mint_nft_with_prediction()

Using Ape's default networks (chains) relies on public RPC nodes that could hit request limits and make the execution of the contract fail. For a better experience think about using a private RPC node with higher quotas

What we have learned

In this tutorial, we learned how to create an AI Agent to mint an MNIST NFT. We created an AI Agent using Ape's framework and interacted with the agent to mint an NFT based on the prediction result of an MNIST image.

We learned how to load and preprocess the MNIST image, create an instance of the agent, predict the MNIST image using the agent, access the prediction result, and mint an NFT based on the prediction result.