diff --git a/docs/concepts/trading-on-vega/data-sources.md b/docs/concepts/trading-on-vega/data-sources.md index 8de2c2d5b..90a7f9f90 100644 --- a/docs/concepts/trading-on-vega/data-sources.md +++ b/docs/concepts/trading-on-vega/data-sources.md @@ -4,9 +4,11 @@ title: Data sourcing hide_title: false --- +import NetworkParameter from '@site/src/components/NetworkParameter'; + Vega's data sourcing framework enables the Vega protocol to acquire and consume data, for example, to settle a market, or to terminate trading at a market's expiry. -It accepts data from several source types, including Ethereum, off-chain data signed by a known key, and data from Vega itself. +It accepts data from several source types, including Ethereum mainnet, EVM chains that support Ethereum RPC calls, off-chain data signed by a known key, and data from Vega itself. The data sourcing framework can also process data from specific, selected fields out of a larger data object, filtering out irrelevant or potentially erroneous data. @@ -23,6 +25,7 @@ For a market proposer looking to choose which data source is best for their mark ## Sources of data Inputs to the data sourcing framework can come from: +* Any chain or layer 2 chain that supports Ethereum RPC calls and runs an EVM * Ethereum smart contracts * Signed message data source, part of the [Open Oracle ↗](https://github.com/compound-finance/open-oracle) feed * Specially formatted and signed JSON messages @@ -46,27 +49,30 @@ Data sources must be able to emit the following data types: * Date/Time - to compare against in filters * Structured data records - such as a set of key and value pairs (inputs to filters) -## Ethereum data sources -Ethereum oracles bridge Ethereum based data sources such as [Chainlink ↗](https://chain.link/) and [UMA ↗](https://uma.xyz/) in to Vega, enabling markets to be settled or priced using data that is verified on Ethereum. The interface is flexible enough that it allows for any Ethereum smart contract to be used as a data source. +## EVM data sources +EVM oracles bridge Ethereum- or any EVM-based data sources in to Vega, enabling markets to be settled or priced using data that is verified on a chain that supports Ethereum RPC calls. Data can only be read from a smart contract based on a timed trigger. -:::note -Currently data can only be read from a smart contract based on a timed trigger. In future Vega will also support events emitted by contracts. -::: +Each chain needs to be supported in the network parameter: +, and by the validators running the network. + +Use a [network parameter proposal](../../tutorials/proposals/network-parameter-proposal.md) to get a new data source chain added to the network. + +Validators can set up their nodes to support the chains by following the [config instructions](../../node-operators/get-started/setup-validator.md#support-evm-chains-for-oracle-data). -When the contract call is triggered by the data source, Vega validator nodes read the selected data from Ethereum and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. +When the contract call is triggered by the data source, Vega validator nodes read the selected data from the chain and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. -An Ethereum data source specification must include: +An EVM data source specification must include: -- Ethereum contract address +- Contract address - ABI in JSON format for the contract (or a subset, covering the parts relevant to fetching data) - Name of the function call, along with any parameters that must be passed through This data can be used for filters, or used as the oracle data itself. -All data sourced from Ethereum is structured as an object containing both a payload and Ethereum chain metadata. Specifically: +All data sourced from Ethereum and EVM chains is structured as an object containing both a payload and chain metadata. Specifically: -- Ethereum block height at which the data was observed/event occurred -- Ethereum block timestamp when the data was observed/event occurred +- Block height at which the data was observed/event occurred +- Block timestamp when the data was observed/event occurred ## Signed message data sources Signed message data sources are a source of off-chain data. They introduce a Vega transaction that represents a data result that is validated by ensuring the signed message is provided by the Vega or Ethereum public key provided in the market’s proposal. @@ -160,12 +166,12 @@ Data source filters allow the market proposer to specify, for a given "root" dat Products on Vega use data to drive actions like settlement and to progress through the market lifecycle. The two key transitions controlled by data sources for cash settled futures are the termination of trading, and settlement of the market. For cash settled perpetuals markets, the data source provides prices for the settlement schedule. These are configured when the market is proposed by providing a data source specification that covers the root source and any filters required to select the specific data or trigger event. -As an Ethereum contract or a public key may provide many messages, a filter is used to extract the required message - for example trading on a futures market could terminate at a specific date and time, and so the filters would ensure that only data provided on or after the specified date and time would trigger termination. +As a smart contract or a public key may provide many messages, a filter is used to extract the required message - for example trading on a futures market could terminate at a specific date and time, and so the filters would ensure that only data provided on or after the specified date and time would trigger termination. When a market is proposed, the market proposal must include details for filters to be applied to the chosen data source(s). Those filters are applied to the source of structured data records that are used as input and determine how data is emitted: such as the specific value for a named field, to return `BTCUSD_PRICE` from a record containing many prices, for example; or price data on/after a certain time. ## Submitting data for a market -When using an Ethereum oracle, there is no need to submit data once the Ethereum oracle is specified in the market proposal. +When using an EVM oracle, there is no need to submit data once the EVM oracle is specified in the market proposal. For other data source types, any Vega keypair can submit settlement and market termination data to the chain. The creator of an instrument for a market has chosen in advance a price source, which data fields the market requires to settle and terminate, and filters that determine when the data is used. diff --git a/docs/concepts/vega-chain/data-nodes.md b/docs/concepts/vega-chain/data-nodes.md index dca10ec51..196e93749 100644 --- a/docs/concepts/vega-chain/data-nodes.md +++ b/docs/concepts/vega-chain/data-nodes.md @@ -10,7 +10,7 @@ import Topic from '/docs/topics/_topic-data-nodes.mdx' -Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transations for the chain and ensuring correctness. +Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transactions for the chain and ensuring correctness. Any processing that isn't required to make the next block is done by the data node. The data node collects, stores, and relates the events, and makes them available through API queries that can be used directly, and through dapps and other tools. diff --git a/docs/node-operators/get-started/setup-validator.md b/docs/node-operators/get-started/setup-validator.md index 92662ed32..75d02ea7b 100644 --- a/docs/node-operators/get-started/setup-validator.md +++ b/docs/node-operators/get-started/setup-validator.md @@ -180,6 +180,30 @@ When a Vega validator node is watching for Ethereum events it will call the `eth PollEventRetryDuration = "20s" ``` +## Support EVM chains for oracle data +To support markets receiving oracle data from EVM chains, your node **must** specify RPC credentials in the Vega config for *Gnosis* and *Arbitrum One* chains. + +Some RPC providers include: + +- [Blast ↗](https://blastapi.io/) - 40 calls/sec(12,000,000 calls/month) +- [OnFinality ↗](https://onfinality.io/) - 500,000 calls/day (15,000,000 calls/month) +- [Ankr ↗](https://ankr.com/) - 30 calls/sec +- [Chainnodes ↗](https://chainnodes.org/) - 25 calls/sec (12,500,000 calls/month) + +```title="YOUR_VEGA_CONFIG/config/node/config.toml" +[Ethereum] + ... + ... + + [[Ethereum.EVMChainConfigs]] + ChainID = "100" + RPCEndpoint = "YOUR_RPC_ENDPOINT_FOR_GNOSIS" + [[Ethereum.EVMChainConfigs]] + ChainID = "42161" + RPCEndpoint = "YOUR_RPC_ENDPOINT_FOR_ARBITRUM_ONE" +... +``` + ## Set up the node wallet Each validator node requires two cryptographic wallets to operate properly: * Ethereum wallet: Used to sign transactions going through the ERC20 bridge diff --git a/docs/tutorials/assets-tokens/withdrawing-assets.md b/docs/tutorials/assets-tokens/withdrawing-assets.md index d07642260..b25bf43da 100644 --- a/docs/tutorials/assets-tokens/withdrawing-assets.md +++ b/docs/tutorials/assets-tokens/withdrawing-assets.md @@ -85,7 +85,7 @@ You can use the following transaction template (formatted for Linux/MacOS): ### Query for withdrawal ID -1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). +1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). You could, instead, calculate it from the transaction signature by generating a sha3 hash in the format: @@ -97,7 +97,7 @@ Once the withdrawal delay has passed, you can carry on with the following steps. ### Retrieve signature bundle -2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-withdrawal.api.mdx). You'll need the withdrawal ID from the step above to do this. +2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-erc-20-withdrawal-approval.api.mdx). You'll need the withdrawal ID from the step above to do this. ### Run asset withdrawal diff --git a/docs/tutorials/proposals/new-market-proposal.md b/docs/tutorials/proposals/new-market-proposal.md index fc34ff379..963464450 100644 --- a/docs/tutorials/proposals/new-market-proposal.md +++ b/docs/tutorials/proposals/new-market-proposal.md @@ -98,7 +98,6 @@ An instrument contains the following properties: | `dataSourceSpecForTradingTermination` | The fields that define the data source used for terminating trading on the market. | vegaprotocol.builtin.timestamp | | `dataSourceSpecBinding` | The fields describe how specific information provided by the data source is used. For example, they can identify the specific name of the settlement price output, or the specific name of the trading termination property. | - For easy reading, the data source filters are separated out - see [Data source bindings](#data-source-bindings) below to see the fields for specifying data. ### Data source bindings @@ -119,6 +118,7 @@ Data source bindings include the following properties: | `conditions` | A filter for the data. The conditions that need to be matched by the data to be considered. This is an optional set of fields. For example you could use an operator and a value to denote that a price should be greater than zero | | `operator` | This adds a constraint to the value, such as LESS_THAN, GREATER_THAN. For example if you wanted to ensure that the price would always be above zero, you would set the operator to ‘GREATER_THAN’ and the Value to be ‘0’ | GREATER_THAN | | `value` | A number that is constrained by the operator. If providing a timestamp, use the Unix time in seconds | 0 | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | :::info Submitting data Learn how to find and submit data in the [submitting data sources tutorial](../using-data-sources.md). @@ -150,7 +150,6 @@ The mark price methodology can be fine-tuned per market. If left blank, the mark | `sourceStalenessTolerance` | How long a price source is considered valid. This uses one entry for each data source, such that the first is for the trade-based mark price, the second is for the order book-based price, and the third is for the first oracle, followed by any other data source staleness tolerance. | 1m0s | | `compositePriceType` | Weighted, median or last trade. | Weighted: Composite price is calculated as a weighted average of the underlying mark prices. Median: Composite price is calculated as a median of the underlying mark prices. Last trade: Composite price is calculated as the last trade price. | - ``` "markPriceConfiguration": { "decayWeight": "1", diff --git a/docs/tutorials/proposals/new-perpetuals-market.md b/docs/tutorials/proposals/new-perpetuals-market.md index 5d1f8e6b1..96cab1646 100644 --- a/docs/tutorials/proposals/new-perpetuals-market.md +++ b/docs/tutorials/proposals/new-perpetuals-market.md @@ -123,6 +123,7 @@ Data source specs include the following properties under `ethOracle`: | `dataSourceSpecBinding` | Describes which property of the data source data is to be used as settlement data and when. | | | `settlementDataProperty` | Name of the property in the source data to be used as settlement data. | `btc.price` | | `settlementScheduleProperty` | Describes what to use to determine when to run a settlement. | `vegaprotocol.builtin.timetrigger` | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | :::info Submitting data Learn how to find and submit data in the [submitting data sources tutorial](../using-data-sources.md). diff --git a/docs/tutorials/using-data-sources.md b/docs/tutorials/using-data-sources.md index a4133ecb5..4f60c9a6e 100644 --- a/docs/tutorials/using-data-sources.md +++ b/docs/tutorials/using-data-sources.md @@ -9,13 +9,13 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ## Choosing a data source when proposing a market -A market proposal must specify details about the data it requires in the market creation governance proposal. When configuring a market's instrument, you will need to select the data sourcefor two events: settlement and trading termination. +A market proposal must specify details about the data it requires. When configuring a market's instrument, you will need to select the data source for the following events: settlement and, if applicable, trading termination. This is done by: 1. Defining a data source spec binding for settlement price 2. Configuring a data source spec for settlement price values -3. Defining a data source spec binding for trading termination -4. Configuring a data source spec for trading termination values +3. Defining a data source spec binding for trading termination, for dated futures markets +4. Configuring a data source spec for trading termination values, for dated futures markets The **binding** tells the market which field contains the value. The **spec** defines where this data will come from, and which values to pass through to the binding. @@ -24,13 +24,16 @@ The **binding** tells the market which field contains the value. The **spec** de * [Tutorial: Proposing a market](./proposals/new-market-proposal.md) ::: -## Ethereum oracles -Since version 0.73.0, settlement data can be sourced from Ethereum smart contracts. The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: +## EVM data sources +Settlement data can be sourced from smart contracts and EVM chains that support Ethereum RPC calls. + +The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: ```javascript "dataSourceSpecForSettlementData": { "external": { "ethOracle": { + "sourceChainId": "1", "address": "0x1b44F3514812d835EB1BDB0acB33d3fA3351Ee43", "abi": "[{\"inputs\":[],\"name\":\"latestRoundData\",\"outputs\":[{\"internalType\":\"int256\",\"name\":\"\",\"type\":\"int256\"}],\"stateMutability\":\"view\",\"type\":\"function\"}]", "method": "latestRoundData", @@ -63,10 +66,12 @@ Since version 0.73.0, settlement data can be sourced from Ethereum smart contrac } ``` -This will cause the network validators to read the result from the specified smart contract and submit the result to Vega. When the data is verified by enough validators, this price is accepted on to the network. This is unlike the other two Oracle types ([Open Oracle signed messages](#open-oracle-signed-messages) & [JSON signed message data](#json-signed-message-data)),which both rely on a third party submitting data to the network. +This will cause the network validators to read the result from the specified smart contract and submit the result to Vega. When the data is verified by enough validators, this price is accepted on to the network. + +This is unlike the other two Oracle types ([Open Oracle signed messages](#open-oracle-signed-messages) & [JSON signed message data](#json-signed-message-data)),which both rely on a third party submitting data to the network. ### ABI -The `address` field in the specification tells the spec above which address to interact with on Ethereum. The `abi` ([Application Binary Interface](https://docs.soliditylang.org/en/develop/abi-spec.html#json) & `method` field on the spec above tells the settlement spec **how** to interact with it. Ethereum Oracle settlement specifications use the JSON ABI of the smart contract to describe the method on the contract that will be called to fetch the data. The ABI will contain the function name, details of any paramters required, and the format of the response. +The `address` field in the specification tells the spec above which address to interact with on the chain. The `abi` ([Application Binary Interface](https://docs.soliditylang.org/en/develop/abi-spec.html#json) & `method` field on the spec above tells the settlement spec **how** to interact with it. EVM oracle settlement specifications use the JSON ABI of the smart contract to describe the method on the contract that will be called to fetch the data. The ABI will contain the function name, details of any paramters required, and the format of the response. For example, the [Chainlink BTC/USD oracle](https://data.chain.link/ethereum/mainnet/crypto-usd/btc-usd) has its JSON ABI [published on Etherscan](https://etherscan.io/address/0xf4030086522a5beea4988f8ca5b36dbc97bee88c#code). When defining the data source spec, you can populate the `abi` field with the full ABI, and then set the `method` to `latestRoundData`. @@ -75,15 +80,15 @@ When populating the `abi` field on your data source spec, you can remove the met ::: ### Time trigger -As it says above, with Ethereum data source specs the validators will read the specified smart contract and method detailed in the ABI. The `trigger` instructs the validators when to do this. In the smple above, it will be called every 30 seconds. +As it says above, with EVM data source specs the validators will read the specified smart contract and method detailed in the ABI. The `trigger` instructs the validators when to do this. In the smple above, it will be called every 30 seconds. ### Normaliser A [JSONPath](https://datatracker.ietf.org/wg/jsonpath/about/) expression use to extract data from the JSON returned from the method call. In the spec above, `expression` is set to `$[0]`, which returns the first item in an array. `$` would return the complete result. ## Open Oracle signed messages -Vega's Data Sourcing framework supports signed ABI-encoded [Open Oracle ↗](https://github.com/compound-finance/open-oracle) or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum. +Vega's data sourcing framework supports signed ABI-encoded [Open Oracle ↗](https://github.com/compound-finance/open-oracle) or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum. -When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and terminate. +When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and optionally terminate. ### Using Open Oracle signed messages in a market proposal For the binding, use the `name` field of the data. In the case of Open Oracle messages, the price data will be availableas 'prices.currency-code.value', for example:`"prices.BTC.value"`. @@ -208,7 +213,7 @@ The [Oracle Data list REST endpoint](../api/rest/data-v2/trading-data-service-li ## JSON signed message data [JSON ↗](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON) messages are a simpler, more configurable alternative to Open Oracle data. They can be totally custom objects, as long as they are valid JSON. As they are not attested by any off-chain source in the way that Open Oracle messages are, and so it's generally advisable to check for an Open Oracle price source before choosing JSON data. The Vega key that signs the message will be referred to as the source for the data. -When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and terminate. +When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and optionally terminate. ### Using JSON signed message data in a market proposal For the binding, use the `name` field of the data. In the following example, the market is settled based on the number of people who have walked on the moon. diff --git a/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md b/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md index 4531909c4..37af14c24 100644 --- a/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md +++ b/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md @@ -10,7 +10,7 @@ import Topic from '/docs/topics/_topic-data-nodes.mdx' -Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transations for the chain and ensuring correctness. +Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transactions for the chain and ensuring correctness. Any processing that isn't required to make the next block is done by the data node. The data node collects, stores, and relates the events, and makes them available through API queries that can be used directly, and through dapps and other tools. diff --git a/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md b/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md index ef9abdc23..66efe488c 100644 --- a/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md +++ b/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md @@ -84,7 +84,7 @@ You can use the following transaction template (formatted for Linux/MacOS): ### Query for withdrawal ID -1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). +1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). You could, instead, calculate it from the transaction signature by generating a sha3 hash in the format: @@ -96,7 +96,7 @@ Once the withdrawal delay has passed, you can carry on with the following steps. ### Retrieve signature bundle -2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-withdrawal.api.mdx). You'll need the withdrawal ID from the step above to do this. +2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-erc-20-withdrawal-approval.api.mdx). You'll need the withdrawal ID from the step above to do this. ### Run asset withdrawal