From 88666dd1c532bad14222e2f8408e50ae21b55943 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Mon, 5 Feb 2024 16:51:53 +0000 Subject: [PATCH 1/9] evm data sourcing --- docs/concepts/trading-on-vega/data-sources.md | 16 +++++++++++----- .../tutorials/proposals/new-market-proposal.md | 1 + .../proposals/new-perpetuals-market.md | 1 + docs/tutorials/using-data-sources.md | 18 +++++++++++------- .../concepts/trading-on-vega/data-sources.md | 2 +- 5 files changed, 25 insertions(+), 13 deletions(-) diff --git a/docs/concepts/trading-on-vega/data-sources.md b/docs/concepts/trading-on-vega/data-sources.md index 8de2c2d5b..b1129ea16 100644 --- a/docs/concepts/trading-on-vega/data-sources.md +++ b/docs/concepts/trading-on-vega/data-sources.md @@ -4,9 +4,11 @@ title: Data sourcing hide_title: false --- +import NetworkParameter from '@site/src/components/NetworkParameter'; + Vega's data sourcing framework enables the Vega protocol to acquire and consume data, for example, to settle a market, or to terminate trading at a market's expiry. -It accepts data from several source types, including Ethereum, off-chain data signed by a known key, and data from Vega itself. +It accepts data from several source types, including Ethereum mainnet, Ethereum L2s that support Ethereum RPC calls, off-chain data signed by a known key, and data from Vega itself. The data sourcing framework can also process data from specific, selected fields out of a larger data object, filtering out irrelevant or potentially erroneous data. @@ -23,6 +25,7 @@ For a market proposer looking to choose which data source is best for their mark ## Sources of data Inputs to the data sourcing framework can come from: +* Any chain or layer 2 chain that supports Ethereum RPC calls and runs an EVM * Ethereum smart contracts * Signed message data source, part of the [Open Oracle ↗](https://github.com/compound-finance/open-oracle) feed * Specially formatted and signed JSON messages @@ -46,14 +49,17 @@ Data sources must be able to emit the following data types: * Date/Time - to compare against in filters * Structured data records - such as a set of key and value pairs (inputs to filters) -## Ethereum data sources -Ethereum oracles bridge Ethereum based data sources such as [Chainlink ↗](https://chain.link/) and [UMA ↗](https://uma.xyz/) in to Vega, enabling markets to be settled or priced using data that is verified on Ethereum. The interface is flexible enough that it allows for any Ethereum smart contract to be used as a data source. +## Ethereum and EVM data sources +Ethereum oracles bridge Ethereum based data sources in to Vega, enabling markets to be settled or priced using data that is verified on Ethereum or a layer 2 chain that supports Ethereum RPC calls. + +Each L2 chain needs to be supported in the network parameter: +, and by the validators running the network. :::note -Currently data can only be read from a smart contract based on a timed trigger. In future Vega will also support events emitted by contracts. +Currently data can only be read from a smart contract based on a timed trigger. ::: -When the contract call is triggered by the data source, Vega validator nodes read the selected data from Ethereum and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. +When the contract call is triggered by the data source, Vega validator nodes read the selected data from the chain and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. An Ethereum data source specification must include: diff --git a/docs/tutorials/proposals/new-market-proposal.md b/docs/tutorials/proposals/new-market-proposal.md index a22e5f6c6..c3703355d 100644 --- a/docs/tutorials/proposals/new-market-proposal.md +++ b/docs/tutorials/proposals/new-market-proposal.md @@ -93,6 +93,7 @@ An instrument contains the following properties: | `dataSourceSpecForSettlementData` | This defines the data source that will be used to identify the settlement price when the market expires. | prices.ORANGES.value | | `dataSourceSpecForTradingTermination` | The fields that define the data source used for terminating trading on the market. | vegaprotocol.builtin.timestamp | | `dataSourceSpecBinding` | The fields describe how specific information provided by the data source is used. For example, they can identify the specific name of the settlement price output, or the specific name of the trading termination property. | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | For easy reading, the data source filters are separated out - see [Data source bindings](#data-source-bindings) below to see the fields for specifying data. diff --git a/docs/tutorials/proposals/new-perpetuals-market.md b/docs/tutorials/proposals/new-perpetuals-market.md index 6461a1875..d694e4235 100644 --- a/docs/tutorials/proposals/new-perpetuals-market.md +++ b/docs/tutorials/proposals/new-perpetuals-market.md @@ -81,6 +81,7 @@ An instrument contains the following properties: | [`dataSourceSpecForSettlementData`](#data-source-for-settlement-data) | This defines the Ethereum data source, the method, normalisers, required confirmations, etc, that will be used to identify the settlement price when the market expires. | | | [`dataSourceSpecForSettlementSchedule`](#data-source-for-settlement-schedule) | This defines how the market will source data for funding, and how often to source it. | | | [`dataSourceSpecBinding`](#data-source-bindings) | The fields describe how specific information provided by the data source is used. For example, they are used to set the settlement data property and the settlement schedule property. | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | ### Data source for settlement schedule The periodic settlements scheduled with the fields below determine how often the market's funding payments occur. It's recommended that funding payments are be less frequent than auction extensions for [price monitoring](#price-monitoring). Very frequent funding payments may lead to quick price changes in the market that participants may not have time to react to. Setting longer funding payment triggers allow for more time. diff --git a/docs/tutorials/using-data-sources.md b/docs/tutorials/using-data-sources.md index 8db96ddbc..7d9983df6 100644 --- a/docs/tutorials/using-data-sources.md +++ b/docs/tutorials/using-data-sources.md @@ -9,13 +9,13 @@ import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; ## Choosing a data source when proposing a market -A market proposal must specify details about the data it requires in the market creation governance proposal. When configuring a market's instrument, you will need to select the data sourcefor two events: settlement and trading termination. +A market proposal must specify details about the data it requires. When configuring a market's instrument, you will need to select the data source for the following events: settlement and, if applicable, trading termination. This is done by: 1. Defining a data source spec binding for settlement price 2. Configuring a data source spec for settlement price values -3. Defining a data source spec binding for trading termination -4. Configuring a data source spec for trading termination values +3. Defining a data source spec binding for trading termination, for dated futures markets +4. Configuring a data source spec for trading termination values, for dated futures markets The **binding** tells the market which field contains the value. The **spec** defines where this data will come from, and which values to pass through to the binding. @@ -25,7 +25,9 @@ The **binding** tells the market which field contains the value. The **spec** de ::: ## Ethereum oracles -Since version 0.73.0, settlement data can be sourced from Ethereum smart contracts. The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: +Settlement data can be sourced from Ethereum smart contracts and L2 chains that support Ethereum RPC calls. + +The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: ```javascript "dataSourceSpecForSettlementData": { @@ -63,7 +65,9 @@ Since version 0.73.0, settlement data can be sourced from Ethereum smart contrac } ``` -This will cause the network validators to read the result from the specified smart contract and submit the result to Vega. When the data is verified by enough validators, this price is accepted on to the network. This is unlike the other two Oracle types ([Open Oracle signed messages](#open-oracle-signed-messages) & [JSON signed message data](#json-signed-message-data)),which both rely on a third party submitting data to the network. +This will cause the network validators to read the result from the specified smart contract and submit the result to Vega. When the data is verified by enough validators, this price is accepted on to the network. + +This is unlike the other two Oracle types ([Open Oracle signed messages](#open-oracle-signed-messages) & [JSON signed message data](#json-signed-message-data)),which both rely on a third party submitting data to the network. ### ABI The `address` field in the specification tells the spec above which address to interact with on Ethereum. The `abi` ([Application Binary Interface](https://docs.soliditylang.org/en/develop/abi-spec.html#json) & `method` field on the spec above tells the settlement spec **how** to interact with it. Ethereum Oracle settlement specifications use the JSON ABI of the smart contract to describe the method on the contract that will be called to fetch the data. The ABI will contain the function name, details of any paramters required, and the format of the response. @@ -83,7 +87,7 @@ A [JSONPath](https://datatracker.ietf.org/wg/jsonpath/about/) expression use to ## Open Oracle signed messages Vega's Data Sourcing framework supports signed ABI-encoded [Open Oracle ↗](https://github.com/compound-finance/open-oracle) or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum. -When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and terminate. +When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and optionally terminate. ### Using Open Oracle signed messages in a market proposal For the binding, use the `name` field of the data. In the case of Open Oracle messages, the price data will be availableas 'prices.currency-code.value', for example:`"prices.BTC.value"`. @@ -208,7 +212,7 @@ The [Oracle Data list REST endpoint](../api/rest/data-v2/trading-data-service-li ## JSON signed message data [JSON ↗](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON) messages are a simpler, more configurable alternative to Open Oracle data. They can be totally custom objects, as long as they are valid JSON. As they are not attested by any off-chain source in the way that Open Oracle messages are, and so it's generally advisable to check for an Open Oracle price source before choosing JSON data. The Vega key that signs the message will be referred to as the source for the data. -When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and terminate. +When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and optionally terminate. ### Using JSON signed message data in a market proposal For the binding, use the `name` field of the data. In the following example, the market is settled based on the number of people who have walked on the moon. diff --git a/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md b/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md index 1f393b815..e96ea6b5b 100644 --- a/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md +++ b/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md @@ -6,7 +6,7 @@ hide_title: false Vega's data sourcing framework enables the Vega protocol to acquire and consume data, for example, to settle a market, or to terminate trading at a market's expiry. -It accepts data from several source types, including Ethereum, off-chain data signed by a known key, and data from Vega itself. +It accepts data from several source types, including Ethereum and EVM chains, off-chain data signed by a known key, and data from Vega itself. The data sourcing framework can also process data from specific, selected fields out of a larger data object, filtering out irrelevant or potentially erroneous data. From 63726a95a1529aa1d5cc4457ee677fbb931fda65 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Thu, 8 Feb 2024 14:36:59 +0000 Subject: [PATCH 2/9] evm sources --- docs/concepts/trading-on-vega/data-sources.md | 20 +++++++++---------- docs/tutorials/using-data-sources.md | 11 +++++----- 2 files changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/concepts/trading-on-vega/data-sources.md b/docs/concepts/trading-on-vega/data-sources.md index b1129ea16..4574f4e77 100644 --- a/docs/concepts/trading-on-vega/data-sources.md +++ b/docs/concepts/trading-on-vega/data-sources.md @@ -49,11 +49,11 @@ Data sources must be able to emit the following data types: * Date/Time - to compare against in filters * Structured data records - such as a set of key and value pairs (inputs to filters) -## Ethereum and EVM data sources -Ethereum oracles bridge Ethereum based data sources in to Vega, enabling markets to be settled or priced using data that is verified on Ethereum or a layer 2 chain that supports Ethereum RPC calls. +## EVM data sources +EVM oracles bridge Ethereum- or any EVM-based data sources in to Vega, enabling markets to be settled or priced using data that is verified on a chain that supports Ethereum RPC calls. Each L2 chain needs to be supported in the network parameter: -, and by the validators running the network. +, and by the validators running the network. Use a [network parameter proposal](../../tutorials/proposals/network-parameter-proposal.md) to get a new data source chain added to the network. :::note Currently data can only be read from a smart contract based on a timed trigger. @@ -61,18 +61,18 @@ Currently data can only be read from a smart contract based on a timed trigger. When the contract call is triggered by the data source, Vega validator nodes read the selected data from the chain and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. -An Ethereum data source specification must include: +An EVM data source specification must include: -- Ethereum contract address +- Contract address - ABI in JSON format for the contract (or a subset, covering the parts relevant to fetching data) - Name of the function call, along with any parameters that must be passed through This data can be used for filters, or used as the oracle data itself. -All data sourced from Ethereum is structured as an object containing both a payload and Ethereum chain metadata. Specifically: +All data sourced from Ethereum and EVM chains is structured as an object containing both a payload and chain metadata. Specifically: -- Ethereum block height at which the data was observed/event occurred -- Ethereum block timestamp when the data was observed/event occurred +- Block height at which the data was observed/event occurred +- Block timestamp when the data was observed/event occurred ## Signed message data sources Signed message data sources are a source of off-chain data. They introduce a Vega transaction that represents a data result that is validated by ensuring the signed message is provided by the Vega or Ethereum public key provided in the market’s proposal. @@ -166,12 +166,12 @@ Data source filters allow the market proposer to specify, for a given "root" dat Products on Vega use data to drive actions like settlement and to progress through the market lifecycle. The two key transitions controlled by data sources for cash settled futures are the termination of trading, and settlement of the market. For cash settled perpetuals markets, the data source provides prices for the settlement schedule. These are configured when the market is proposed by providing a data source specification that covers the root source and any filters required to select the specific data or trigger event. -As an Ethereum contract or a public key may provide many messages, a filter is used to extract the required message - for example trading on a futures market could terminate at a specific date and time, and so the filters would ensure that only data provided on or after the specified date and time would trigger termination. +As a smart contract or a public key may provide many messages, a filter is used to extract the required message - for example trading on a futures market could terminate at a specific date and time, and so the filters would ensure that only data provided on or after the specified date and time would trigger termination. When a market is proposed, the market proposal must include details for filters to be applied to the chosen data source(s). Those filters are applied to the source of structured data records that are used as input and determine how data is emitted: such as the specific value for a named field, to return `BTCUSD_PRICE` from a record containing many prices, for example; or price data on/after a certain time. ## Submitting data for a market -When using an Ethereum oracle, there is no need to submit data once the Ethereum oracle is specified in the market proposal. +When using an EVM oracle, there is no need to submit data once the EVM oracle is specified in the market proposal. For other data source types, any Vega keypair can submit settlement and market termination data to the chain. The creator of an instrument for a market has chosen in advance a price source, which data fields the market requires to settle and terminate, and filters that determine when the data is used. diff --git a/docs/tutorials/using-data-sources.md b/docs/tutorials/using-data-sources.md index 7d9983df6..e8b61cfa2 100644 --- a/docs/tutorials/using-data-sources.md +++ b/docs/tutorials/using-data-sources.md @@ -24,8 +24,8 @@ The **binding** tells the market which field contains the value. The **spec** de * [Tutorial: Proposing a market](./proposals/new-market-proposal.md) ::: -## Ethereum oracles -Settlement data can be sourced from Ethereum smart contracts and L2 chains that support Ethereum RPC calls. +## EVM data sources +Settlement data can be sourced from smart contracts and L2 chains that support Ethereum RPC calls. The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: @@ -33,6 +33,7 @@ The following spec would read from the Ethereum contract at `0x1b4...e43` every "dataSourceSpecForSettlementData": { "external": { "ethOracle": { + "sourceChainId": "1", "address": "0x1b44F3514812d835EB1BDB0acB33d3fA3351Ee43", "abi": "[{\"inputs\":[],\"name\":\"latestAnswer\",\"outputs\":[{\"internalType\":\"int256\",\"name\":\"\",\"type\":\"int256\"}],\"stateMutability\":\"view\",\"type\":\"function\"}]", "method": "latestAnswer", @@ -70,7 +71,7 @@ This will cause the network validators to read the result from the specified sma This is unlike the other two Oracle types ([Open Oracle signed messages](#open-oracle-signed-messages) & [JSON signed message data](#json-signed-message-data)),which both rely on a third party submitting data to the network. ### ABI -The `address` field in the specification tells the spec above which address to interact with on Ethereum. The `abi` ([Application Binary Interface](https://docs.soliditylang.org/en/develop/abi-spec.html#json) & `method` field on the spec above tells the settlement spec **how** to interact with it. Ethereum Oracle settlement specifications use the JSON ABI of the smart contract to describe the method on the contract that will be called to fetch the data. The ABI will contain the function name, details of any paramters required, and the format of the response. +The `address` field in the specification tells the spec above which address to interact with on the chain. The `abi` ([Application Binary Interface](https://docs.soliditylang.org/en/develop/abi-spec.html#json) & `method` field on the spec above tells the settlement spec **how** to interact with it. EVM oracle settlement specifications use the JSON ABI of the smart contract to describe the method on the contract that will be called to fetch the data. The ABI will contain the function name, details of any paramters required, and the format of the response. For example, the [Chainlink BTC/USD oracle](https://data.chain.link/ethereum/mainnet/crypto-usd/btc-usd) has its JSON ABI [published on Etherscan](https://etherscan.io/address/0xf4030086522a5beea4988f8ca5b36dbc97bee88c#code). When defining the data source spec, you can populate the `abi` field with the full ABI, and then set the `method` to `latestAnswer`. @@ -79,13 +80,13 @@ When populating the `abi` field on your data source spec, you can remove the met ::: ### Time trigger -As it says above, with Ethereum data source specs the validators will read the specified smart contract and method detailed in the ABI. The `trigger` instructs the validators when to do this. In the smple above, it will be called every 30 seconds. +As it says above, with EVM data source specs the validators will read the specified smart contract and method detailed in the ABI. The `trigger` instructs the validators when to do this. In the smple above, it will be called every 30 seconds. ### Normaliser A [JSONPath](https://datatracker.ietf.org/wg/jsonpath/about/) expression use to extract data from the JSON returned from the method call. In the spec above, `expression` is set to `$[0]`, which returns the first item in an array. `$` would return the complete result. ## Open Oracle signed messages -Vega's Data Sourcing framework supports signed ABI-encoded [Open Oracle ↗](https://github.com/compound-finance/open-oracle) or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum. +Vega's data sourcing framework supports signed ABI-encoded [Open Oracle ↗](https://github.com/compound-finance/open-oracle) or JSON messages. ABI-encoded signed messages can be verified to have come from the public key that signed them, which allows markets on Vega to use pricing data sourced from Ethereum. When it's time for a market to settle, someone needs to submit the data that matches the data source spec defined in the market. Any Vega keypair can submit the data. In the configuration for a market, a data source specification field dictates which data feeds it is interested in. In effect, it works as a filter. This specification means that the creator of an instrument for a market will choose in advance a price source, and which data fields the market requires to settle and optionally terminate. From 49420cb677e5541b4cbea64c0b7f2e316d921382 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Thu, 8 Feb 2024 15:31:42 +0000 Subject: [PATCH 3/9] evm sources --- docs/concepts/trading-on-vega/data-sources.md | 4 ++-- docs/tutorials/using-data-sources.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/concepts/trading-on-vega/data-sources.md b/docs/concepts/trading-on-vega/data-sources.md index 4574f4e77..a8cc5d2b6 100644 --- a/docs/concepts/trading-on-vega/data-sources.md +++ b/docs/concepts/trading-on-vega/data-sources.md @@ -8,7 +8,7 @@ import NetworkParameter from '@site/src/components/NetworkParameter'; Vega's data sourcing framework enables the Vega protocol to acquire and consume data, for example, to settle a market, or to terminate trading at a market's expiry. -It accepts data from several source types, including Ethereum mainnet, Ethereum L2s that support Ethereum RPC calls, off-chain data signed by a known key, and data from Vega itself. +It accepts data from several source types, including Ethereum mainnet, EVM chains that support Ethereum RPC calls, off-chain data signed by a known key, and data from Vega itself. The data sourcing framework can also process data from specific, selected fields out of a larger data object, filtering out irrelevant or potentially erroneous data. @@ -52,7 +52,7 @@ Data sources must be able to emit the following data types: ## EVM data sources EVM oracles bridge Ethereum- or any EVM-based data sources in to Vega, enabling markets to be settled or priced using data that is verified on a chain that supports Ethereum RPC calls. -Each L2 chain needs to be supported in the network parameter: +Each chain needs to be supported in the network parameter: , and by the validators running the network. Use a [network parameter proposal](../../tutorials/proposals/network-parameter-proposal.md) to get a new data source chain added to the network. :::note diff --git a/docs/tutorials/using-data-sources.md b/docs/tutorials/using-data-sources.md index e8b61cfa2..5e9a6e89a 100644 --- a/docs/tutorials/using-data-sources.md +++ b/docs/tutorials/using-data-sources.md @@ -25,7 +25,7 @@ The **binding** tells the market which field contains the value. The **spec** de ::: ## EVM data sources -Settlement data can be sourced from smart contracts and L2 chains that support Ethereum RPC calls. +Settlement data can be sourced from smart contracts and EVM chains that support Ethereum RPC calls. The following spec would read from the Ethereum contract at `0x1b4...e43` every 30 seconds, and fetch the Bitcoin price value from the returned object: From df2afc8ef544f3d1d1512a4136f878d93c5e7e18 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Thu, 8 Feb 2024 17:23:31 +0000 Subject: [PATCH 4/9] data sources --- .../version-v0.73/concepts/trading-on-vega/data-sources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md b/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md index e96ea6b5b..1f393b815 100644 --- a/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md +++ b/versioned_docs/version-v0.73/concepts/trading-on-vega/data-sources.md @@ -6,7 +6,7 @@ hide_title: false Vega's data sourcing framework enables the Vega protocol to acquire and consume data, for example, to settle a market, or to terminate trading at a market's expiry. -It accepts data from several source types, including Ethereum and EVM chains, off-chain data signed by a known key, and data from Vega itself. +It accepts data from several source types, including Ethereum, off-chain data signed by a known key, and data from Vega itself. The data sourcing framework can also process data from specific, selected fields out of a larger data object, filtering out irrelevant or potentially erroneous data. From f5ff30173e78439eb2cf7b9e1afc253d34cd1a56 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Tue, 13 Feb 2024 12:42:50 +0000 Subject: [PATCH 5/9] evm info for validators --- docs/concepts/trading-on-vega/data-sources.md | 10 ++++---- .../get-started/setup-validator.md | 24 +++++++++++++++++++ 2 files changed, 29 insertions(+), 5 deletions(-) diff --git a/docs/concepts/trading-on-vega/data-sources.md b/docs/concepts/trading-on-vega/data-sources.md index a8cc5d2b6..90a7f9f90 100644 --- a/docs/concepts/trading-on-vega/data-sources.md +++ b/docs/concepts/trading-on-vega/data-sources.md @@ -50,14 +50,14 @@ Data sources must be able to emit the following data types: * Structured data records - such as a set of key and value pairs (inputs to filters) ## EVM data sources -EVM oracles bridge Ethereum- or any EVM-based data sources in to Vega, enabling markets to be settled or priced using data that is verified on a chain that supports Ethereum RPC calls. +EVM oracles bridge Ethereum- or any EVM-based data sources in to Vega, enabling markets to be settled or priced using data that is verified on a chain that supports Ethereum RPC calls. Data can only be read from a smart contract based on a timed trigger. Each chain needs to be supported in the network parameter: -, and by the validators running the network. Use a [network parameter proposal](../../tutorials/proposals/network-parameter-proposal.md) to get a new data source chain added to the network. +, and by the validators running the network. -:::note -Currently data can only be read from a smart contract based on a timed trigger. -::: +Use a [network parameter proposal](../../tutorials/proposals/network-parameter-proposal.md) to get a new data source chain added to the network. + +Validators can set up their nodes to support the chains by following the [config instructions](../../node-operators/get-started/setup-validator.md#support-evm-chains-for-oracle-data). When the contract call is triggered by the data source, Vega validator nodes read the selected data from the chain and submit a transaction that includes the filtered data. When that data is verified by enough validators, the market's data source specification then acts on the submitted data. diff --git a/docs/node-operators/get-started/setup-validator.md b/docs/node-operators/get-started/setup-validator.md index 92662ed32..75d02ea7b 100644 --- a/docs/node-operators/get-started/setup-validator.md +++ b/docs/node-operators/get-started/setup-validator.md @@ -180,6 +180,30 @@ When a Vega validator node is watching for Ethereum events it will call the `eth PollEventRetryDuration = "20s" ``` +## Support EVM chains for oracle data +To support markets receiving oracle data from EVM chains, your node **must** specify RPC credentials in the Vega config for *Gnosis* and *Arbitrum One* chains. + +Some RPC providers include: + +- [Blast ↗](https://blastapi.io/) - 40 calls/sec(12,000,000 calls/month) +- [OnFinality ↗](https://onfinality.io/) - 500,000 calls/day (15,000,000 calls/month) +- [Ankr ↗](https://ankr.com/) - 30 calls/sec +- [Chainnodes ↗](https://chainnodes.org/) - 25 calls/sec (12,500,000 calls/month) + +```title="YOUR_VEGA_CONFIG/config/node/config.toml" +[Ethereum] + ... + ... + + [[Ethereum.EVMChainConfigs]] + ChainID = "100" + RPCEndpoint = "YOUR_RPC_ENDPOINT_FOR_GNOSIS" + [[Ethereum.EVMChainConfigs]] + ChainID = "42161" + RPCEndpoint = "YOUR_RPC_ENDPOINT_FOR_ARBITRUM_ONE" +... +``` + ## Set up the node wallet Each validator node requires two cryptographic wallets to operate properly: * Ethereum wallet: Used to sign transactions going through the ERC20 bridge From 03e3f9a0d54c7942e1215d02bcc5f8963d7c6950 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Tue, 13 Feb 2024 14:35:27 +0000 Subject: [PATCH 6/9] data nodes --- docs/concepts/vega-chain/data-nodes.md | 2 +- versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/concepts/vega-chain/data-nodes.md b/docs/concepts/vega-chain/data-nodes.md index dca10ec51..196e93749 100644 --- a/docs/concepts/vega-chain/data-nodes.md +++ b/docs/concepts/vega-chain/data-nodes.md @@ -10,7 +10,7 @@ import Topic from '/docs/topics/_topic-data-nodes.mdx' -Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transations for the chain and ensuring correctness. +Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transactions for the chain and ensuring correctness. Any processing that isn't required to make the next block is done by the data node. The data node collects, stores, and relates the events, and makes them available through API queries that can be used directly, and through dapps and other tools. diff --git a/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md b/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md index 4531909c4..37af14c24 100644 --- a/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md +++ b/versioned_docs/version-v0.73/concepts/vega-chain/data-nodes.md @@ -10,7 +10,7 @@ import Topic from '/docs/topics/_topic-data-nodes.mdx' -Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transations for the chain and ensuring correctness. +Protocol users need to see and interact with data, such as price history, market changes, validator scores, and more. While the core emits events when states change, it does not store the data about those events. The core is responsible for processing transactions for the chain and ensuring correctness. Any processing that isn't required to make the next block is done by the data node. The data node collects, stores, and relates the events, and makes them available through API queries that can be used directly, and through dapps and other tools. From f0ac998a169f4d0459546901d36b16f069a0a62a Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Thu, 15 Feb 2024 16:39:19 +0000 Subject: [PATCH 7/9] mark price config --- .../proposals/new-market-proposal.md | 30 +++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/docs/tutorials/proposals/new-market-proposal.md b/docs/tutorials/proposals/new-market-proposal.md index 80be27edc..76b36ebf0 100644 --- a/docs/tutorials/proposals/new-market-proposal.md +++ b/docs/tutorials/proposals/new-market-proposal.md @@ -138,6 +138,36 @@ Liquidity monitoring uses the following properties: | `timeWindow` | Defines the length of time (in seconds) over which open interest is measured. | 3600 | | `scalingFactor` | The target stake scaling factor scales the estimated required liquidity (based on the market's risk model and current market data) to yield the market's target stake. The scaling factor must be a number greater than zero and finite | 10 | +### Mark price configuration +The mark price methodology can be fine-tuned per market. If left blank, the market will default to the [last price method](../../concepts/trading-on-vega/margin.md#last-traded-price). You can read further details about the flexible mark price fields in [concepts](../../concepts/trading-on-vega/margin.md#flexible-mark-price-methodology). + +| Field | Description | Value | +| ----------- | ----------- | --------- | +| `decayWeight` | Controls to what extent observation time impacts the weight in the mark price calculation. 0 implies uniform weights. | 1 | +| `decayPower` | Controls how quickly the weight assigned to older observations should drop. The higher the value, the more weight is assigned to recent observations. | 1 | +| `cashAmount` | Used in calculating the mark price from the order book, in asset decimals. Use the margin amount of the expected typical trade size, at maximum leverage. | A well-known highly liquid exchange uses 200 USDT on their most popular market. If you expect your market will be equally liquid, use the equivalent amount in the market's settlement asset. If you think it's likely to be 10x less liquid, use 10x less. | +| `sourceWeights` | Determines how much weight goes to each composite price component. The order of sources used is as follows: price by trades, price by book, oracle_1, ... oracle_n, median price. 0 means the input is always ignored.| 0.5, 0.5, 0 uses an average of trades as defined via the TWAP and decay, and book as defined by the cash amount.| +| `sourceStalenessTolerance` | How long a price source is considered valid. This uses one entry for each data source, such that the first is for the trade-based mark price, the second is for the order book-based price, and the third is for the first oracle, followed by any other data source staleness tolerance. | 1m0s | +| `compositePriceType` | Weighted, median or last trade. | Weighted: Composite price is calculated as a weighted average of the underlying mark prices. Median: Composite price is calculated as a median of the underlying mark prices. Last trade: Composite price is calculated as the last trade price. | + +``` +"markPriceConfiguration": { + "decayWeight": "1", + "decayPower": "1", + "cashAmount": "2000000", + "sourceWeights": [ + "0.5", + "0.5", + "0" + ], + "sourceStalenessTolerance": [ + "1m0s", + "1m0s", + "1m0s" + ], + "compositePriceType": "COMPOSITE_PRICE_TYPE_WEIGHTED", +``` + ### Price monitoring Price monitoring parameters are optional, and configure the acceptable price movement bounds for price monitoring. If you leave these blank, they will default to the value of the network parameter ). See below for more details on each field. From 937776e70e355225e373e371dd4d3041329f8613 Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Thu, 15 Feb 2024 17:11:11 +0000 Subject: [PATCH 8/9] source chain --- docs/tutorials/proposals/new-market-proposal.md | 3 +-- docs/tutorials/proposals/new-perpetuals-market.md | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/docs/tutorials/proposals/new-market-proposal.md b/docs/tutorials/proposals/new-market-proposal.md index 76b36ebf0..1016cb711 100644 --- a/docs/tutorials/proposals/new-market-proposal.md +++ b/docs/tutorials/proposals/new-market-proposal.md @@ -97,8 +97,6 @@ An instrument contains the following properties: | `dataSourceSpecForSettlementData` | This defines the data source that will be used to identify the settlement price when the market expires. | prices.ORANGES.value | | `dataSourceSpecForTradingTermination` | The fields that define the data source used for terminating trading on the market. | vegaprotocol.builtin.timestamp | | `dataSourceSpecBinding` | The fields describe how specific information provided by the data source is used. For example, they can identify the specific name of the settlement price output, or the specific name of the trading termination property. | -| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | - For easy reading, the data source filters are separated out - see [Data source bindings](#data-source-bindings) below to see the fields for specifying data. @@ -120,6 +118,7 @@ Data source bindings include the following properties: | `conditions` | A filter for the data. The conditions that need to be matched by the data to be considered. This is an optional set of fields. For example you could use an operator and a value to denote that a price should be greater than zero | | `operator` | This adds a constraint to the value, such as LESS_THAN, GREATER_THAN. For example if you wanted to ensure that the price would always be above zero, you would set the operator to ‘GREATER_THAN’ and the Value to be ‘0’ | GREATER_THAN | | `value` | A number that is constrained by the operator. If providing a timestamp, use the Unix time in seconds | 0 | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | :::info Submitting data Learn how to find and submit data in the [submitting data sources tutorial](../using-data-sources.md). diff --git a/docs/tutorials/proposals/new-perpetuals-market.md b/docs/tutorials/proposals/new-perpetuals-market.md index a4ed40fa6..8c0029645 100644 --- a/docs/tutorials/proposals/new-perpetuals-market.md +++ b/docs/tutorials/proposals/new-perpetuals-market.md @@ -84,7 +84,6 @@ An instrument contains the following properties: | [`dataSourceSpecForSettlementData`](#data-source-for-settlement-data) | This defines the Ethereum data source, the method, normalisers, required confirmations, etc, that will be used to identify the settlement price when the market expires. | | | [`dataSourceSpecForSettlementSchedule`](#data-source-for-settlement-schedule) | This defines how the market will source data for funding, and how often to source it. | | | [`dataSourceSpecBinding`](#data-source-bindings) | The fields describe how specific information provided by the data source is used. For example, they are used to set the settlement data property and the settlement schedule property. | -| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | ### Data source for settlement schedule The periodic settlements scheduled with the fields below determine how often the market's funding payments occur. It's recommended that funding payments are be less frequent than auction extensions for [price monitoring](#price-monitoring). Very frequent funding payments may lead to quick price changes in the market that participants may not have time to react to. Setting longer funding payment triggers allow for more time. @@ -124,6 +123,7 @@ Data source specs include the following properties under `ethOracle`: | `dataSourceSpecBinding` | Describes which property of the data source data is to be used as settlement data and when. | | | `settlementDataProperty` | Name of the property in the source data to be used as settlement data. | `btc.price` | | `settlementScheduleProperty` | Describes what to use to determine when to run a settlement. | `vegaprotocol.builtin.timetrigger` | +| `sourceChainId` | Describes the chain ID of the data source. This chain must already be enabled in network parameters and supported by validators. | :::info Submitting data Learn how to find and submit data in the [submitting data sources tutorial](../using-data-sources.md). From e00bb87e3cc07da1a66dfdf800afae9d1a91ab7a Mon Sep 17 00:00:00 2001 From: candida-d <62548908+candida-d@users.noreply.github.com> Date: Mon, 19 Feb 2024 13:42:18 +0000 Subject: [PATCH 9/9] fix endpoint --- docs/tutorials/assets-tokens/withdrawing-assets.md | 4 ++-- .../tutorials/assets-tokens/withdrawing-assets.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/tutorials/assets-tokens/withdrawing-assets.md b/docs/tutorials/assets-tokens/withdrawing-assets.md index d07642260..b25bf43da 100644 --- a/docs/tutorials/assets-tokens/withdrawing-assets.md +++ b/docs/tutorials/assets-tokens/withdrawing-assets.md @@ -85,7 +85,7 @@ You can use the following transaction template (formatted for Linux/MacOS): ### Query for withdrawal ID -1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). +1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). You could, instead, calculate it from the transaction signature by generating a sha3 hash in the format: @@ -97,7 +97,7 @@ Once the withdrawal delay has passed, you can carry on with the following steps. ### Retrieve signature bundle -2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-withdrawal.api.mdx). You'll need the withdrawal ID from the step above to do this. +2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-erc-20-withdrawal-approval.api.mdx). You'll need the withdrawal ID from the step above to do this. ### Run asset withdrawal diff --git a/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md b/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md index ef9abdc23..66efe488c 100644 --- a/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md +++ b/versioned_docs/version-v0.73/tutorials/assets-tokens/withdrawing-assets.md @@ -84,7 +84,7 @@ You can use the following transaction template (formatted for Linux/MacOS): ### Query for withdrawal ID -1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). +1. Query for your withdrawal ID, using the [list withdrawals API](../../api/rest/data-v2/trading-data-service-list-withdrawals.api.mdx). You could, instead, calculate it from the transaction signature by generating a sha3 hash in the format: @@ -96,7 +96,7 @@ Once the withdrawal delay has passed, you can carry on with the following steps. ### Retrieve signature bundle -2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-withdrawal.api.mdx). You'll need the withdrawal ID from the step above to do this. +2. Retrieve the signature bundle using the [get withdrawal API](../../api/rest/data-v2/trading-data-service-get-erc-20-withdrawal-approval.api.mdx). You'll need the withdrawal ID from the step above to do this. ### Run asset withdrawal