djWebdApp Tezos

Indexing contracts

Example contract

We will need to instanciate a contract on this blockchain. We’ll use a simple example that looks like some FA12, in Pure Python (rather than “Smart” Python), thanks to Python to Michelson compiler by Thomas Binetruy-Pic, based on Python AST, Pytezos, and an original idea by your favorite open source zealot:

from dataclasses import dataclass
from pymich.michelson_types import Address, BigMap, Contract, Nat
from pymich.stdlib import SENDER


@dataclass
class FA12(Contract):
    tokens: BigMap[Address, Nat]
    total_supply: Nat
    owner: Address

    def mint(self, _to: Address, value: Nat):
        if SENDER != self.owner:
            raise Exception("Only owner can mint")

        self.total_supply = self.total_supply + value

        self.tokens[_to] = self.tokens.get(_to, Nat(0)) + value

    def transfer(self, _from: Address, _to: Address, value: Nat):
        from_balance = self.tokens.get(_from, Nat(0))

        if (from_balance - value) < Int(0):
            raise Exception("NotEnoughBalance")

        self.tokens[_from] = abs(from_balance - value)

        to_balance = self.tokens.get(_to, Nat(0))

        self.tokens[_to] = to_balance + value

    def getTotalSupply(self) -> Nat:  # noqa
        return self.total_supply

We already compiled it, but you can change it and recompile it with the following command:

pip install pymich
cd src/djwebdapp_example/tezos
pymich FA12.py FA12.json

Local tezos blockchain

Danger

Before you begin, make sure you have followed the setup instructions from Local blockchains.

Sandbox ids are predefined and hardcoded, you can find them in the tezos-init-sandboxed-client.sh script.

Let’s deploy our example contract using pytezos, first install pytezos and start a python shell with the ./manage.py shell command at the root of our repository:

pip install pytezos
./manage.py shell

Note

The above example also works in a normal Python shell started with the python command, but we need to be in the Django shell of the demo project to go through this tutorial anyway.

In the shell, do the following to have a pytezos client with a sandbox account:

# use local blockchain with sandbox account
from pytezos import pytezos
client = pytezos.using(
    key='edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh',
    shell='http://tzlocal:8732',
)

Check your client balance:

>> client.account()
{'balance': '3997440000000', 'delegate': 'tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx', 'counter': '0'}

Deploy a smart contract

First, load the smart contract source code:

# load contract source code
import json
source = json.load(open('src/djwebdapp_example/tezos/FA12.json'))

# initial storage in micheline :P
storage = {
    'prim': 'Pair',
    'args': [
        [],
        {'int': '0'},
        {'string': 'tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx'}
    ]}

Let’s deploy our smart contract and call the mint() entrypoint by pasting the following in our pytezos python shell started above, which you need to start if you haven’t already to run the following commands:

# originate contract source code with given storage, wait 1 confirmation block
opg = client.origination(
    dict(code=source, storage=storage)
).send(min_confirmations=1)

# get originated contract address
from pytezos.operation.result import OperationResult
res = OperationResult.from_operation_group(opg.opg_result)
address = res[0].originated_contracts[0]

# let's mint some sweet tokens
client.contract(address).mint(
    'tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx',
    1000,
).send(min_confirmations=2)

This should store the deployed contract address in the address variable, copy it or leave the shell open because you need it to index the contract in the next section.

Setting up a blockchain network

Now that we have deployed a contract, let’s setup djwebdapp for a local ethereum node, also programatically in ./manage.py shell:

# First, we need to add a blockchain in the database
from djwebdapp.models import Blockchain
blockchain, _ = Blockchain.objects.get_or_create(
    name='Tezos Local',
    provider_class='djwebdapp_tezos.provider.TezosProvider',
)

# Add our node to the blockchain
blockchain.node_set.get_or_create(endpoint='http://tzlocal:8732')

Indexing a contract

Now that we have deployed a contract, and setup djwebdapp for a local tezos node, let’s index a contract, also programatically in ./manage.py shell:

# Insert a smart contract with our address
from djwebdapp_tezos.models import TezosTransaction
contract = TezosTransaction.objects.create(
    blockchain=blockchain,
    address=address,
)

# Transaction kind was setup automatically
assert contract.kind == 'contract'

# Unspecified, micheline was downloaded automatically
assert contract.micheline

# But calls have not yet been synchronized
assert not contract.call_set.count()

# Let's index the blockchain, you could also run ./manage.py index
blockchain.provider.index()

# Refresh our contract model object
contract.refresh_from_db()

# Gas cost was indexed
assert contract.gas

# Mint call was indexed
call = contract.call_set.first()
assert call.function == 'mint'
assert call.args['value'] == 1000

Normalizing incomming data: Models

We have created example models in the src/djwebdapp_example directory:

"""
Example models to demonstrate features of djwebdapp.
"""


from django.db import models


class FA12(models.Model):
    """
    Model representing an FA12 contract on at least one blockchain.

    .. py:attribute:: name

        The name of the FA12 token, ie.: "Your New Token"

    .. py:attribute:: symbol

        Symbol of the FA12 token, ie.: "YNT"

    .. note:: You wouldn't need to have relations to contracts on every
              blockchain, but we have them here so that we are later able to
              demonstrate inter-blockchain mirroring.
    """
    tezos_contract = models.OneToOneField(
        'djwebdapp_tezos.TezosTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    ethereum_contract = models.OneToOneField(
        'djwebdapp_ethereum.EthereumTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    name = models.CharField(
        max_length=200,
        null=True,
        blank=True,
    )
    symbol = models.CharField(
        max_length=10,
        null=True,
        blank=True,
    )


class Mint(models.Model):
    """
    Model representing a mint() call on an FA12 contract.

    .. py:attribute:: address

        Recipient address for the mint.

    .. py:attribute:: value

        Amount of tokens minted.

    .. note:: You wouldn't need to have relations to contract calls on every
              blockchain, but we have them here so that we are later able to
              demonstrate inter-blockchain mirroring.
    """
    fa12 = models.ForeignKey(
        'FA12',
        on_delete=models.CASCADE,
    )
    tezos_call = models.OneToOneField(
        'djwebdapp_tezos.TezosTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    ethereum_call = models.OneToOneField(
        'djwebdapp_ethereum.EthereumTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    address = models.ForeignKey(
        'djwebdapp.Account',
        on_delete=models.CASCADE,
    )
    value = models.PositiveIntegerField()

    def __str__(self):
        return f'mint({self.address.address}, {self.value})'


class Balance(models.Model):
    """
    Model representing the balance of an address on an FA12 token.
    """
    fa12 = models.ForeignKey(
        'FA12',
        on_delete=models.CASCADE,
    )
    address = models.ForeignKey(
        'djwebdapp.Account',
        on_delete=models.CASCADE,
        related_name='fa12_balance_set',
    )
    balance = models.PositiveIntegerField()

    class Meta:
        unique_together = (
            ('fa12', 'address'),
        )

    def __str__(self):
        return f'{self.address} balance: {self.balance}'


# importing other documentation code snippets concerning signals here
from .ethereum.mint_normalize import mint_normalize_ethereum  # noqa
from .tezos.mint_normalize import mint_normalize_tezos        # noqa

Note

You wouldn’t have to declare ForeignKeys to other Transaction classes than TezosTransactions, but we’ll learn to do inter-blockchain mirroring later in this tutorial, so that’s why we have relations to both.

And declared a function to update the balance of an FA12 contract:

from django.db.models import Q, Sum


def balance_update(fa12, address):
    """
    Account balance calculator.

    We completely recalculate the balance here so that we are able to keep
    correct results even after a blockchain reorg.

    Note that the blockchain implementation is completely out of the way here:
    we're dealing with normalized models!
    """

    # calculate a balance total
    total = fa12.mint_set.exclude(
        # exclude calls deleted by reorg!!
        Q(tezos_call__state='deleted') | Q(ethereum_call__state='deleted')
    ).filter(
        address=address,
    ).aggregate(
        total=Sum('value')
    )['total']

    # you would have to add burn() and transfer() method support here if you
    # had them in your smart contract

    # set the balance for an address
    address.fa12_balance_set.update_or_create(
        fa12=fa12,
        defaults=dict(
            balance=total,
        ),
    )

Normalizing incomming data: Signals

Finally, to connect the dots, we are first going to connect a custom callback to djwebdapp_tezos.models.TezosTransaction’s post_save signal to create normalized Mint objects for every mint() call we index:

from django.db.models import signals
from django.dispatch import receiver

from djwebdapp_tezos.models import TezosTransaction
from djwebdapp_example.balance_update import balance_update
from djwebdapp_example.models import FA12


@receiver(signals.post_save, sender=TezosTransaction)
def mint_normalize_tezos(sender, instance, **kwargs):
    if instance.function != 'mint':
        # not a mint call? bail out!
        return

    try:
        fa12 = instance.contract.fa12
    except TezosTransaction.fa12.RelatedObjectDoesNotExist:
        # no FA12 normalized object for this contract? bail out!
        return

    # figure out the beneficiary Account based on the mint call arg _to
    beneficiary = instance.blockchain.account_set.get(
        address=instance.args['_to'],
    )

    # create or update the normalized Mint object for this call
    fa12.mint_set.update_or_create(
        tezos_call=instance,
        defaults=dict(
            address=beneficiary,
            value=instance.args['value'],
        )
    )

    # we're fully recalculating the balance here in case of a blockchain reorg
    # to ensure the balance is always current
    balance_update(fa12, beneficiary)


@receiver(signals.post_save, sender=FA12)
def fa12_create_tezos(sender, instance, created, **kwargs):
    """
    Trigger post_save on every call that the contract already has, if any.
    """
    if not created:
        # we're setup already
        return

    if not instance.tezos_contract:
        # not a tezos contract
        return

    for call in instance.tezos_contract.call_set.all():
        call.save()

We are now ready to normalize the smart contract we have indexed:

# Create a normalized FA12 model for this Tezos contract
from djwebdapp_example.models import FA12
fa12 = FA12.objects.create(tezos_contract=contract)

# reverse relation works as usual
assert contract.fa12

# mint calls were normalized
assert contract.fa12.mint_set.count() == 1

# balance was calculated
from djwebdapp_example.models import Balance
assert Balance.objects.first().balance == 1000

Vault

Setup

Make sure you have installed djwebdapp with the [vault] dependencies (or [all]).

Note

You may rotate Fernet keys used for encryption, please refer to djfernet documentation.

Importing a wallet

# import a bootstrap wallet by secret key
from djwebdapp.models import Account
from pytezos import Key

# Use update_or_create, in case we already have this address!
import binascii
bootstrap, _ = Account.objects.update_or_create(
    blockchain=blockchain,
    address='tz1KqTpEZ7Yob7QbPE4Hy4Wo8fHG8LhKxZSx',
    defaults=dict(
        secret_key=binascii.b2a_base64(Key.from_encoded_key(
            'edsk3gUfUPyBSfrS9CCgmCiQsTCHGkviBDusMxDJstFtojtc1zcpsh'
        ).secret_exponent).decode(),
    ),
)

# balance was automatically fetched
assert bootstrap.balance > 0
old_balance = bootstrap.balance

Creating a wallet

# create a new wallet on that blockchain, secret key auto generates
new_wallet = Account.objects.create(blockchain=blockchain)
assert new_wallet.balance == 0

Transfering coins

# Use the transaction model with an amount argument to transfer coins
from djwebdapp.models import Transaction
transaction = Transaction.objects.create(
    name='Provision 1.000 coins',
    amount=1_000,
    sender=bootstrap,
    receiver=new_wallet,
    blockchain=blockchain,
)

# Deploy the transaction now
transaction.deploy()

# Wait one block level
transaction.blockchain.wait(transaction.level + 1)

blockchain.provider.index()

new_wallet.refresh_balance()

Refreshing balances

# we can also refresh balances of all accounts with this method
# you would rather have ./manage.py refresh_balances in a cron or something
new_wallet.refresh_balance()
assert new_wallet.balance == 1_000
assert new_wallet.provider.get_balance() == new_wallet.balance

Deploy a smart contract

from djwebdapp_tezos.models import TezosTransaction

# Create a smart contract to deploy
contract = TezosTransaction.objects.create(
    sender=bootstrap,
    state='deploy',
    max_fails=2,  # try twice before aborting, to speed up tests!
    micheline=source,
    args=storage,
)

# Create a call that should deploy afterwards on that contract
mint = TezosTransaction.objects.create(
    sender=bootstrap,
    state='deploy',
    max_fails=2,
    contract=contract,
    function='mint',
    args=(
        new_wallet.address,
        1000,
    ),
)

# Spool will first deploy the contract
assert blockchain.provider.spool() == contract

# Get deployment level
contract.refresh_from_db()

# But won't deploy the mint call on the same block
assert not blockchain.provider.spool()

# Waiting for next block ...
blockchain.wait(contract.level + 1)

# Index to fetch contract address
blockchain.provider.index()

# Now spool will deploy the mint call!
assert blockchain.provider.spool() == mint

mint.refresh_from_db()

blockchain.wait(mint.level + 1)

blockchain.provider.index()