¶ Overview
Walletd is a HTTP server which provides JSON 2.0 RPC interface for payment operations and address management. It allows to create a container of multiple addresses under a single parent wallet. This service is useful for ecommerce platforms, exchanges, and web wallet providers.
¶ Setup
To start using RPC Walletd you must first generate a container. Container file is the only file that stores all data required to run your service. It contains user addresses and private keys required to operate them. Make sure to backup this file regularly.
To generate a new container you should run the following command:
./walletd --generate-container --container-file <container-file> --container-password <container-password>
Where
- container-file is the container file name and a path to it (relative or absolute); path is optional in this argument, specifying only a container's name will result in new file located in the same folder as RPC Walletd. If a file exists with same name, it won't be overwritten; and
- container-password is a secret password for the new container file.
If the operation is successful the master address is provided as a response and the process exits.
¶ Start RPC Wallet
RPC Walletd requires connection to a Quan node to operates. This connection can be provided as a node daemon running locally or remotely, or by an in-process which means RPC Walletd will run a self contained node without the need to run a separated daemon.
¶ Use a Remote Node Daemon
Remote connection allows you to bind your Quan RPC Walletd to a remote daemon - quand -. You may deploy daemon on either local or remote machines and connect to. Such type of connection allows you to start RPC Walletd on a relatively slow machine while heavy loaded daemon is going to work on a separate powerful server.
- for local daemons use 127.0.0.1 as daemon IP address.
- for remote daemons use the remote daemon's IP address.
Default Quan daemon ports are 8313 (P2P) and 8314 (RPC).
Use the following command to start Quan RPC Walletd with a remote connection:
./walletd --container-file <container-file> --container-password <container-password> --daemon-address <daemon-ip> --daemon-port <daemon-port> --rpc-user <rpc-username> --rpc-password <rpc-password>
Where
- container-file is the container file name and a path to it (relative or absolute); path is optional in this argument, specifying only a container's name will result in new file located in the same folder as RPC Walletd;
- container-password is a secret password for the container file;
- daemon-ip is the IP address of Quan daemon quand. Currently dns name are not supported;
- daemon-port is the RPC port of Quan daemon quand. Default RPC daemon port is 8314;
- rpc-user is the credential's user required for any API requests; and
- rpc-password is the credential's password required for any API requests.
¶ Use in-Process Node
You can also start RPC Wallet with an in-process node. This allows you to start RPC Wallet out-of-box with no external daemon required. You will get a fully functional node for Quan network inside RPC Wallet. You don't have to download or install anything besides RPC Wallet. This approach will help reduce the overheads required for infrastructure maintenance.
Use the following command to start RPC Walletd with an in-process node:
./walletd --container-file=<container-file> --container-password=<container-password> --rpc-user <rpc-username> --rpc-password <rpc-password> --local
Where
- container-file is the container file name and a path to it (relative or absolute);
- container-password is a secret password for the container file;
- rpc-user is the credential's user required for any API requests; and
- rpc-password is the credential's password required for any API requests.
¶ API Methods
To make a JSON PRC request to your Quan RPC Wallet you should use POST request that looks like this:
http://<ip-address>:<port>/json_rpc
Walletd, by default, listen on port 8070.
The following methods are available in Quan RPC Walletd:
reset | reset() method re-synchronizes wallet's data.
save | save() method saves wallet state.
getViewKey | getViewKey() method returns address view key.
getSpendKeys | getSpendKeys() method returns address spend keys.
getStatus | getStatus() method returns information about the current RPC Wallet state: block_count, known_block_count, last_block_hash and peer_count.
getAddresses | getAddresses() method returns an array of your RPC Wallet's addresses.
createAddress | createAddress() method creates an address.
deleteAddress | deleteAddress() method deletes a specified address.
validateAdress | validateAddress method checks if a wallet address is valid for the Quan blockchain.
getBalance | getBalance() method returns a balance for a specified address. If address is not specified, returns a cumulative balance of all RPC Wallet's addresses.
getBlockHashes | getBlockHashes() method returns an array of block hashes for a specified block range.
getTransactionHashes | getTransactionHashes() method returns an array of block and transaction hashes.
getTransactions | getTransactions() method returns information about the transactions in specified block range or for specified addresses.
getUnconfirmedTransactionHashes | getUnconfirmedTransactionHashes() method returns information about the current unconfirmed transaction pool or for a specified addresses.
getTransaction | getTransaction() method returns information about the specified transaction.
sendTransaction | sendTransaction() method creates and sends a transaction.
createDelayedTransaction | createDelayedTransaction() method creates but not sends a transaction.
getDelayedTransactionHashes | getDelayedTransactionHashes() method returns hashes of delayed transactions.
deleteDelayedTransaction | deleteDelayedTransaction() method deletes a specified delayed transaction.
sendDelayedTransaction | sendDelayedTransaction() method sends a specified delayed transaction.
sendFusionTransaction | sendFusionTransaction() method creates and sends a fusion transaction.
estimateFusion | estimateFusion() method allows to estimate a number of outputs that can be optimized with fusion transactions.
Click on method name to see details.
¶ How to Protect Walletd Container and Addresses
It is important to have data protection strategies for your container. This procedure alone is not enough to give full protection. Traditional data backup processes are still needed as well as frequent restore simulations.
¶ Backup
To be able to recover an addresses container, you need to have:
- The viewSecretKey of the container;
- The spendSecretKey of all addresses on your container.
The viewSecretKey of the container can be obtained using the method getViewKey. See the example below.
Request:
curl -X POST -i -H "Accept: application/json" -d '{"params": {}, "jsonrpc": "2.0", "method": "getViewKey"}' http://127.0.0.1:8070/json_rpc
Result:
{"jsonrpc":"2.0","result":{"viewSecretKey":"8aa4c6356aed9145cf6e52a0b583798e3b94809bca97820e154ec620a77d2d0c"}}
To obtain the spendSecretKey of an address you use the method getSpendKeys.
Request:
curl -X POST -i -H "Accept: application/json" -d '{"params": {"address": "N9KoD6bkLMrG5y1THonmu25MFVvQJt7rLF5wY57Jbd5yh33vbEwfnvFfcqfnbhZXtLNNS7qHacySfPpQN5B2dsmz9dMsFEj"}, "jsonrpc": "2.0", "method": "getSpendKeys"}' http://127.0.0.1:8070/json_rpc
Result:
{"jsonrpc":"2.0","result":{"spendPublicKey":"68a9a64b1c0f5d5a30e5338b8e58611a0075485066f6ad5435b398d195e7d0ef","spendSecretKey":"1422830f555e181fbf4fdb65928d74ce0ed02f2dbad449f257a31a7848d6d501"}}
Run this method for every new address created and save it in a safe place together with the others spendSecretKey's.
¶ Restore
To recover your container, first generate a brand new one.
./walletd --generate-container --container-file myContainer --container-password myContainer --local
Then start it.
./walletd --container-file myContainer --container-password myContainer --local
Now, on another terminal, call the reset method passing the viewSecretKey you have saved before.
curl -X POST -i -H "Accept: application/json" -d '{"params": {"viewSecretKey":"8aa4c6356aed9145cf6e52a0b583798e3b94809bca97820e154ec620a77d2d0c"}, "jsonrpc": "2.0", "method": "reset"}' http://127.0.0.1:8070/json_rpc
This call gives an empty JSON value as result.
And finally, for every spendSecretKey from your backup, recreate the original address using the method createAddress.
Request:
curl -X POST -i -H "Accept: application/json" -d '{"params": {"spendSecretKey":"1422830f555e181fbf4fdb65928d74ce0ed02f2dbad449f257a31a7848d6d501"}, "jsonrpc": "2.0", "method": "createAddress"}' http://127.0.0.1:8070/json_rpc
Result:
{"jsonrpc":"2.0","result":{"address":"N9KoD6bkLMrG5y1THonmu25MFVvQJt7rLF5wY57Jbd5yh33vbEwfnvFfcqfnbhZXtLNNS7qHacySfPpQN5B2dsmz9dMsFEj"}}
Click on method name to see details.
¶ Optimization Tool for RPC Wallets
We developed an optimizer utility for wallets in a RPC container (walletd). This is an original work from Quan developers.
¶ Motivation
Cryptonote derived codes, such as Quan, use the concept of transaction inputs and transaction outputs to create a transfer of coins from one wallet to another. This often leads to a problem where the transaction being built becomes too large and is rejected by the code. The common workaround is to break the transfer into smaller ones. There is a special kind of transaction, called fusion transaction, that can be used to optimize these entries and prevent the problem. We'll dig on the details below.
Transaction inputs, or simply tx inputs can be seem as the money bills someone have on his pocket. Let's take Carol as example. At some point in time other person or persons give Carol this money and she now keeps them on her pocket. These money bills are the so called tx inputs which, summed up, represent the total amount available she has to spend at this moment. If Carol wants to pay someone a certain amount of money, she must select a group of money bills that compose a value equal or bigger than that required amount, and she must be able to hold them all on her hands.
On Cryptonote's blockchain, a similar process is used. A transfer between wallets is made by a transaction, which in turn contains a group of tx inputs that represents part (or total) of the funds of the sender, and the tx output locking that amount to the receiver of the payment. The tx output of a transaction will become the tx input for a future transaction, when the receiver of the former transfer will act as a sender.
¶ The Problem #1
Now imagine a situation where Carol holds a lot of one cent coins and needs to pay two thousand dollars to a friend. She has the amount, but her hands won't be able to hold all these coins at once. On real world this is a bizarre situation, but for cryptocurrencies on a blockchain it's a common case. Many wallets receive thousands of small value payments from mining or from faucets, and when their owners need to transfer some amount the transaction becomes too big to be created (don't fit on their hands).
There is another factor that contributes to an even bigger transaction. We have a system of privacy that mix copies of unrelated tx inputs with the transaction, creating a diversion mechanism to protect the identity of the sender of the coins. Depending on the anonymity factor used, the number of tx inputs used can be ten times the number of the actual inputs, or even more.
¶ The Solution #1
What is being commonly called wallet optimization is a process of changing the small value bills for higher value ones. If Carol can go to the bank and change two thousand dollars, in one cent coins, for twenty bills of a hundred dollars each, she will be able to hold them all together and make the payment she needs to.
This solution is brought to blockchain in the form of the fusion transaction, which differs from the regular transaction on two major points: the sender and the receiver are the same address, and this kind of transaction doesn't pay fee. In other words, Carol sends two thousand dollars to herself.
The code on her wallet will take many small tx inputs and group them into a single tx output. As she acts as the sender and the recipient on this transaction, this tx output can be used by her as a tx input to pay somewhere else, on a next step.
The RPC wallet API, on most Cryptonote currencies, has an endpoint to create such fusion transactions - sendFusionTransaction.
¶ The Problem #2
The RPC Wallet (walletd) is mainly used by exchanges and web wallet providers. A substantial percentage of their support tickets are related to failed withdrawns from their customers. And guess what? Most failures are caused by the transaction too big problem.
So why don't they run fusion transactions?
The answer may differ on details, but they all rely on lack of confidence on this process. It is somehow complex to run this API call on all addresses of the container without writing some kind of script or program to manage it. Walletd is a container of addresses, so first another method must be called to request the list of addresses from the container, then loop them all to call another method. A lot of other details must be addressed on this task, such as error handling, time window for running it, among others.
We can only imagine the exchanges don't want to dig into this matter, mainly because they may not have the technical skills on the specifics of our blockchain code, or maybe they just don't want to invest time from their technical teams to solve other's problem.
¶ The Solution #2
The good news is that Quan developers created a special tool to facilitate the optimization process, bringing safety and a set of options to control the whole activity.
An additional binary tool called optimizer is included on the Quan node package. It allows an easy way to automate the execution, controlling the maximum duration it is allowed to run, the interval between interactions so the local mempool does not get flooded, and some other handy command line options. It is even possible to run the optimizer on a preview mode, where no actual action is taken, but it can be verified the number of wallets that are eligible to be optimized.
¶ The Tool
The utility is called optimizer and can be found on the package of binaries from the Quan node daemon. This is the help page for it:
$ ./optimizer --help
General options:
--help Produce help message
Command options:
--address arg Address of the wallet to optimize inputs. If
not provided, all addresses will be checked
and, if applicable, optimized using polling
interval between each interaction. Default:
All
--walletd-ip arg (=127.0.0.1) IP address of walletd. Default: 127.0.0.1
--walletd-port arg (=8070) RPC port of walletd. Default: 8070
--walletd-user arg RPC user. Default: none
--walletd-password arg RPC password. Default: none
--interval arg polling interval in seconds. Default: 5.
Minimum: 1. Maximum: 120.
--duration arg maximum execution time, in minutes. Default: 0
(unlimited)
--threshold arg Only outputs lesser than the threshold value
will be included into optimization. Default:
100000000000000 (do not use decimal point)
--anonymity arg Privacy level. Higher values give more privacy
but bigger transactions. Default: 6
--preview print on screen what it would be doing, but
not really doing it
The interval is a pause between each interaction, where an interaction is the creation of a fusion transaction for a wallet on the list.
The duration is the maximum time allowed to the tool remains running. The process is stopped after the limit is reached. This is useful to automate the process, making sure it does not extrapolate a maintenance window.
A few recommendations, although not mandatory:
- The fusion transaction will appear on the customer's transaction history as a zero amount transaction. To avoid questioning, it may be useful to make a small change on frontend code to hide these transactions;
- It is probably a good idea to create periodic maintenance windows, once a week for example, where the wallets operations are brought offline for customers, and run the optimizer inside these windows. As any other regular transactions, the coins used stays locked until get into a block, so customer can face a race condition where he will fail to create a transfer attempted on a time frame close to the fusion transaction creation for his account.
Below a image of an execution of the tool, followed by a preview, on a container.
The code for the optimizer tool can be found on github:
https://github.com/quan-projects/quan-node/blob/master/src/Optimizer/Optimizer.cpp