djWebdApp Ethereum

Indexing contracts

Example contract

We will need to instanciate a contract on this blockchain. We’ll use a simple example smart contract in solidity that looks like some FA12:

pragma solidity ^0.8.0;


contract FA12 {
    mapping(address => uint256) private _balances;

    uint256 private _totalSupply;

    string private _name;
    string private _symbol;

    constructor(string memory name_, string memory symbol_) {
        _name = name_;
        _symbol = symbol_;
    }

    function mint(address account, uint256 amount) public virtual {
        require(account != address(0), "ERC20: mint to the zero address");

        _totalSupply += amount;
        _balances[account] += amount;
    }
}

We already compiled it, but you can change it and recompile it with the following command:

cd src/djwebdapp_example/ethereum
solc --abi --overwrite --output-dir . --bin FA12.sol

Example contract deployment

Danger

Before you begin, make sure you have followed the setup instructions from Local blockchains.

With the Ethereum sandbox, we’ll use the default account which is already provisionned with some ethers.

Let’s deploy our example contract using Web3py, install it and start a Python shell with the ./manage.py shell command at the root of our repository:

pip install web3
./manage.py shell

Note

The above example also works in a normal Python shell started with the python command, but we need to be in the Django shell of the demo project to go through this tutorial anyway.

In the shell, make sure your default account is provisionned properly:

from web3 import Web3

# use local blockchain with default account
client = Web3(Web3.HTTPProvider('http://ethlocal:8545'))
client.eth.default_account = client.eth.accounts[0]

# enable support for geth --dev sandbox
from web3.middleware import geth_poa_middleware
client.middleware_onion.inject(geth_poa_middleware, layer=0)

Check your client balance:

>>> client.eth.default_account
'0xD1562e5128FC95311E46129a9f445402278e7751'
>>> client.eth.get_balance(w3.eth.default_account)
115792089237316195423570985008687907853269984665640564039457577993160770347781

Deploy a smart contract

First, load the smart contract source code:

bytecode = open('src/djwebdapp_example/ethereum/FA12.bin', 'r').read()
abi = open('src/djwebdapp_example/ethereum/FA12.abi', 'r').read()

Let’s deploy our smart contract and call the mint() entrypoint by pasting the following in our python shell started above, which you need to start if you haven’t already to run the following commands:

# actually deploy the contract:
contract = client.eth.contract(abi=abi, bytecode=bytecode)
contract_hash = contract.constructor('Your New Token', 'YNT').transact()
receipt = client.eth.wait_for_transaction_receipt(contract_hash)
address = receipt.contractAddress

# let's mint some sweet YNTs
contract = client.eth.contract(abi=abi, address=address)
hash = contract.functions.mint(client.eth.default_account, 1000).transact()
receipt = client.eth.wait_for_transaction_receipt(hash)

This should store the deployed contract address in the address variable, copy it or leave the shell open because you need it to index the contract in the next section.

Setting up a blockchain network

Now that we have deployed a contract, let’s setup djwebdapp for a local ethereum node, also programatically in ./manage.py shell:

# First, we need to add a blockchain in the database
from djwebdapp.models import Blockchain
blockchain, _ = Blockchain.objects.get_or_create(
    name='Ethereum Local',
    provider_class='djwebdapp_ethereum.provider.EthereumProvider',
)

# Add our node to the blockchain
blockchain.node_set.get_or_create(endpoint='http://ethlocal:8545')

Indexing a contract

Now that we have setup djwebdapp for a local ethereum node, let’s index a contract, also programatically in ./manage.py shell:

# Then, insert a smart contract with our address
from djwebdapp_ethereum.models import EthereumTransaction
contract = EthereumTransaction.objects.create(
    blockchain=blockchain,
    # used to index method calls
    address=address,
    # used to translate function calls
    abi=abi,
    # used to fill the contract metadata
    hash=contract_hash.hex(),
)
assert contract.kind == 'contract'

# But calls have not yet been synchronized
assert not contract.call_set.count()

# Let's index the blockchain, you could also run ./manage.py index
blockchain.provider.index()

# Refresh our contract model object
contract.refresh_from_db()

# Gas cost was indexed
assert contract.gas

# Mint call was indexed
call = contract.call_set.first()
assert call.function == 'mint'
assert call.args['amount'] == 1000

Normalizing incomming data: Models

We have created example models in the src/djwebdapp_example directory:

"""
Example models to demonstrate features of djwebdapp.
"""


from django.db import models


class FA12(models.Model):
    """
    Model representing an FA12 contract on at least one blockchain.

    .. py:attribute:: name

        The name of the FA12 token, ie.: "Your New Token"

    .. py:attribute:: symbol

        Symbol of the FA12 token, ie.: "YNT"

    .. note:: You wouldn't need to have relations to contracts on every
              blockchain, but we have them here so that we are later able to
              demonstrate inter-blockchain mirroring.
    """
    tezos_contract = models.OneToOneField(
        'djwebdapp_tezos.TezosTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    ethereum_contract = models.OneToOneField(
        'djwebdapp_ethereum.EthereumTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    name = models.CharField(
        max_length=200,
        null=True,
        blank=True,
    )
    symbol = models.CharField(
        max_length=10,
        null=True,
        blank=True,
    )


class Mint(models.Model):
    """
    Model representing a mint() call on an FA12 contract.

    .. py:attribute:: address

        Recipient address for the mint.

    .. py:attribute:: value

        Amount of tokens minted.

    .. note:: You wouldn't need to have relations to contract calls on every
              blockchain, but we have them here so that we are later able to
              demonstrate inter-blockchain mirroring.
    """
    fa12 = models.ForeignKey(
        'FA12',
        on_delete=models.CASCADE,
    )
    tezos_call = models.OneToOneField(
        'djwebdapp_tezos.TezosTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    ethereum_call = models.OneToOneField(
        'djwebdapp_ethereum.EthereumTransaction',
        on_delete=models.CASCADE,
        null=True,
        blank=True,
    )
    address = models.ForeignKey(
        'djwebdapp.Account',
        on_delete=models.CASCADE,
    )
    value = models.PositiveIntegerField()

    def __str__(self):
        return f'mint({self.address.address}, {self.value})'


class Balance(models.Model):
    """
    Model representing the balance of an address on an FA12 token.
    """
    fa12 = models.ForeignKey(
        'FA12',
        on_delete=models.CASCADE,
    )
    address = models.ForeignKey(
        'djwebdapp.Account',
        on_delete=models.CASCADE,
        related_name='fa12_balance_set',
    )
    balance = models.PositiveIntegerField()

    class Meta:
        unique_together = (
            ('fa12', 'address'),
        )

    def __str__(self):
        return f'{self.address} balance: {self.balance}'


# importing other documentation code snippets concerning signals here
from .ethereum.mint_normalize import mint_normalize_ethereum  # noqa
from .tezos.mint_normalize import mint_normalize_tezos        # noqa

Note

You wouldn’t have to declare ForeignKeys to other Transaction classes than EthereumTransactions, but we’ll learn to do inter-blockchain mirroring later in this tutorial, so that’s why we have relations to both.

And declared a function to update the balance of an FA12 contract:

from django.db.models import Q, Sum


def balance_update(fa12, address):
    """
    Account balance calculator.

    We completely recalculate the balance here so that we are able to keep
    correct results even after a blockchain reorg.

    Note that the blockchain implementation is completely out of the way here:
    we're dealing with normalized models!
    """

    # calculate a balance total
    total = fa12.mint_set.exclude(
        # exclude calls deleted by reorg!!
        Q(tezos_call__state='deleted') | Q(ethereum_call__state='deleted')
    ).filter(
        address=address,
    ).aggregate(
        total=Sum('value')
    )['total']

    # you would have to add burn() and transfer() method support here if you
    # had them in your smart contract

    # set the balance for an address
    address.fa12_balance_set.update_or_create(
        fa12=fa12,
        defaults=dict(
            balance=total,
        ),
    )

Normalizing incomming data: Signals

Finally, to connect the dots, we are first going to connect a custom callback to djwebdapp_ethereum.models.EthereumTransaction’s post_save signal to create normalized Mint objects for every mint() call we index:

from django.db.models import signals
from django.dispatch import receiver

from djwebdapp_ethereum.models import EthereumTransaction
from djwebdapp_example.balance_update import balance_update
from djwebdapp_example.models import FA12


@receiver(signals.post_save, sender=EthereumTransaction)
def mint_normalize_ethereum(sender, instance, **kwargs):
    if instance.function != 'mint':
        # not a mint call? bail out!
        return

    try:
        fa12 = instance.contract.fa12
    except EthereumTransaction.fa12.RelatedObjectDoesNotExist:
        # no FA12 normalized object for this contract? bail out!
        return

    # figure out the beneficiary Account based on the mint call arg _to
    beneficiary = instance.blockchain.account_set.get(
        address=instance.args['account'],
    )

    # create or update the normalized Mint object for this call
    fa12.mint_set.update_or_create(
        ethereum_call=instance,
        defaults=dict(
            address=beneficiary,
            value=instance.args['amount'],
        )
    )

    # we're fully recalculating the balance here in case of a blockchain reorg
    # to ensure the balance is always current
    balance_update(fa12, beneficiary)


@receiver(signals.post_save, sender=FA12)
def fa12_create_ethereum(sender, instance, created, **kwargs):
    """
    Trigger post_save on every call that the contract already has, if any.
    """
    if not created:
        # we're setup already
        return

    if not instance.ethereum_contract:
        # not an ethereum contract
        return

    for call in instance.ethereum_contract.call_set.all():
        call.save()

We are now ready to normalize the smart contract we have indexed:

# Create a normalized FA12 model for this Ethereum contract
from djwebdapp_example.models import FA12
fa12 = FA12.objects.create(ethereum_contract=contract)

# reverse relation works as usual
assert contract.fa12

# mint calls were normalized
assert contract.fa12.mint_set.count() == 1

# balance was calculated
from djwebdapp_example.models import Balance
assert Balance.objects.first().balance == 1000

Vault

Setup

Make sure you have installed djwebdapp with the [vault] dependencies (or [all]).

Note

You may rotate Fernet keys used for encryption, please refer to djfernet documentation.

Importing a wallet

# create a keyfile, we could have created it with geth account new too
new_account = client.eth.account.create()
keyfile = new_account.encrypt('')

# decode private key and get address
address = Web3.toChecksumAddress(keyfile['address'])
private_key = blockchain.provider.client.eth.account.decrypt(keyfile, '')

# send some ether from the seed account
client.eth.send_transaction(dict(
    to=address,
    value=client.toWei(1337, 'ether'),
))

# wait until the blockchain validates the transfer
import time
while not client.eth.get_balance(address):
    time.sleep(.1)

# import the freshly created wallet by secret key
import binascii
from djwebdapp.models import Account
bootstrap = Account.objects.create(
    secret_key=binascii.b2a_base64(private_key).decode(),
    address=address,
    blockchain=blockchain,
)

# balance was automatically fetched
assert bootstrap.balance == 1337
old_balance = bootstrap.balance

Creating a wallet

# create a new wallet on that blockchain, secret key auto generates
new_wallet = Account.objects.create(blockchain=blockchain)
assert new_wallet.balance == 0

Transfering coins

# Use the transaction model with an amount argument to transfer coins
from djwebdapp.models import Transaction
transaction = Transaction.objects.create(
    name='Provision 1.000 coins',
    amount=1_000,
    sender=bootstrap,
    receiver=new_wallet,
    blockchain=blockchain,
)

# Deploy the transaction now
transaction.deploy()

Refreshing balances

# we can also refresh balances of all accounts with this method
# you would rather have ./manage.py refresh_balances in a cron or something
new_wallet.refresh_balance()
assert new_wallet.balance == 1_000
assert new_wallet.provider.get_balance() == new_wallet.balance

Deploy a contract

from djwebdapp_ethereum.models import EthereumTransaction

# Create a smart contract to deploy
contract = EthereumTransaction.objects.create(
    sender=bootstrap,
    state='deploy',
    max_fails=2,  # try twice before aborting, to speed up tests!
    bytecode=bytecode,
    abi=abi,
    args=['Your New Token', 'YNT'],
)

# Create a call that should deploy afterwards on that contract
mint = EthereumTransaction.objects.create(
    sender=bootstrap,
    state='deploy',
    max_fails=2,
    contract=contract,
    function='mint',
    args=(
        new_wallet.address,
        1000,
    ),
)

# Spool will first deploy the contract
assert blockchain.provider.spool() == contract

# Get deployment level
contract.refresh_from_db()

# Geth --dev incremented the block level transactionnaly,
# but you'd need that to wait a block on a real network as such
blockchain.wait(contract.level + 1)

# Now spool will deploy the mint call!
assert blockchain.provider.spool() == mint

# And has nothing else to do
assert not blockchain.provider.spool()