Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimizing competitor kpis #89

Merged
merged 9 commits into from
Dec 13, 2024
Merged

Optimizing competitor kpis #89

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3d12d6e
Changing price from minute to daily to get the latest tvl. It increas…
PoloX2021 Dec 6, 2024
cc2ad7b
Filter the time window for prices.mintue
PoloX2021 Dec 6, 2024
bdacff9
Fixed typos for date and ordering
PoloX2021 Dec 9, 2024
02db83b
Restructuring the curve query to improve efficiency for the curve query
PoloX2021 Dec 9, 2024
2123552
changed the join to where
PoloX2021 Dec 9, 2024
ffe78ba
Additional comments
PoloX2021 Dec 9, 2024
65c9076
Simplified the time comparison, and changed time argument from start …
PoloX2021 Dec 11, 2024
dd3d1f9
Linting
PoloX2021 Dec 11, 2024
20bd708
added a parameter wrongfully removed
PoloX2021 Dec 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 28 additions & 13 deletions cowamm/kpi/competitors/curve/curve_kpis_4232873.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,17 @@
-- Parameters:
-- {{blockchain}}: The blockchain to query
-- {{competitor_end_time}}: The end time of the time window (end_time - 1 day; end_time), defaults to now()
select
contract_address,
tvl,
fee,
volume,
apr
from (

with accumulated_kpis as (
select
contract_address,
r.contract_address,
fee,
tvl,
latest_per_pool,
sum(amount_usd) over (partition by contract_address order by latest_per_pool) as volume,
365 * sum(amount_usd * fee / tvl) over (partition by contract_address order by latest_per_pool) as apr
-- The first call to 4232976 gets the tvl after each tx to compute volume/tvl
from "query_4232976(blockchain='{{blockchain}}')" as r
sum(amount_usd) over (partition by r.contract_address order by latest_per_pool desc) as volume,
365 * sum(amount_usd * fee / (reserve0 * p0.price * power(10, -p0.decimals) + reserve1 * p1.price * power(10, -p1.decimals))) over (partition by r.contract_address order by latest_per_pool desc) as apr,
-- new index to make sure rows don't get lost in the filtering later
row_number() over (partition by r.contract_address order by r.evt_block_time desc) as latest_per_pool
from "query_4232976(blockchain='{{blockchain}}', number_of_pools = '{{number_of_pools}}', end_time = '{{competitor_end_time}}')" as r
left join
( --noqa: ST05
select *
Expand All @@ -29,8 +24,28 @@ from (
on
r.contract_address = t.project_contract_address
and r.tx_hash = t.tx_hash
inner join prices.minute as p0
on
r.token0 = p0.contract_address
and date_trunc('minute', r.evt_block_time) = p0.timestamp
inner join prices.minute as p1
on
r.token1 = p1.contract_address
and date_trunc('minute', r.evt_block_time) = p1.timestamp
where
-- This test avoids any possible issue with reconstructing the reserves of the pool
tvl > 0
and p0.timestamp >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and p0.timestamp <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe using

p0.timestamp between date_add('day', -1, least('{{competitor_end_time}}', now())) and least('{{competitor_end_time}}', now())

is slightly easier to read? Also I'm not sure if reducing the range to now() is actually making the query more effective (since there are no datapoints beyond now the full table scan should complete in the same time)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea!

About the now(), the query is built to only search for 24 hours, so the upper bound is meant to backfill the data base

and p1.timestamp >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and p1.timestamp <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
)

select
contract_address,
fee,
tvl,
volume,
apr
from accumulated_kpis
where latest_per_pool = 1
38 changes: 17 additions & 21 deletions cowamm/kpi/competitors/curve/curve_largest_2token_pools_4232976.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,46 +54,42 @@ reserves as (
token0,
token1,
tx_hash,
block_time,
block_time as evt_block_time,
fee,
sum(transfer0) over (partition by contract_address order by block_time, evt_index) as reserve0,
sum(transfer1) over (partition by contract_address order by block_time, evt_index) as reserve1,
row_number() over (partition by tx_hash, contract_address order by evt_index desc) as latest_per_tx,
row_number() over (partition by contract_address order by block_time desc) as latest_per_pool
row_number() over (partition by contract_address order by block_time desc, evt_index desc) as latest_per_pool
from transfers
where block_time <= (case when '{{end_time}}' = '2100-01-01' then now() else timestamp '{{end_time}}' end)
),

-- finds the TVL of the pools
recent_tvl as (
latest_tvl as (
select
r.contract_address,
token0,
token1,
block_time,
tx_hash,
reserve0,
reserve1,
fee,
latest_per_pool,
(reserve0 * p0.price / pow(10, p0.decimals)) + (reserve1 * p1.price / pow(10, p1.decimals)) as tvl
from reserves as r
inner join prices.minute as p0
--using the daily value to get a better representation of the TVL over the 24 hour period
inner join prices.day as p0
on
date_trunc('minute', block_time) = p0.timestamp
p0.timestamp = date_trunc('day', case when ('{{end_time}}' = '2100-01-01' or date('{{end_time}}') = date_trunc('day', now())) then now() - interval '1' day else date('{{end_time}}') end)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this logic seems unnecessarily complex. Isn't this the same as

least(date('{{end_time}}'), now() - interval '1' day)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also please document the new parameter

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, indeed :)

and token0 = p0.contract_address
inner join prices.minute as p1
inner join prices.day as p1
on
date_trunc('minute', block_time) = p1.timestamp
p1.timestamp = date_trunc('day', case when ('{{end_time}}' = '2100-01-01' or date('{{end_time}}') = date_trunc('day', now())) then now() - interval '1' day else date('{{end_time}}') end)
and token1 = p1.contract_address
where latest_per_tx = 1
)


select * from recent_tvl
where contract_address in (
select contract_address
from recent_tvl
where latest_per_pool = 1
order by tvl desc
limit {{number_of_pools}}
)

select
reserves.*,
tvl
from reserves
inner join latest_tvl
on reserves.contract_address = latest_tvl.contract_address
where latest_per_tx = 1
11 changes: 9 additions & 2 deletions cowamm/kpi/competitors/uni_swap_style/largest_uni_style_kpis_4304295.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ with pool as (
token0,
token1,
tvl
from "query_4303563(blockchain='{{blockchain}}', number_of_pools = '{{number_of_pools}}')"
from "query_4303563(blockchain='{{blockchain}}', number_of_pools = '{{number_of_pools}}', end_time = '{{competitor_end_time}}')"
),

syncs as (
Expand All @@ -31,6 +31,7 @@ syncs as (
on logs.contract_address = pool.contract_address
where
block_time >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and block_time <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
and topic0 = 0x1c411e9a96e071241c2f21f7726b17ae89e3cab4c78be50e062b03a9fffbbad1 -- Sync
),

Expand All @@ -47,7 +48,8 @@ swaps as (
varbinary_to_uint256(substr(data, 97, 32)) as amount1Out
from {{blockchain}}.logs
where
block_time >= date(date_add('day', -1, now()))
block_time >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and block_time <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
and topic0 = 0xd78ad95fa46c994b6551d0da85fc275fe613ce37657fb8d5e3d130840159d822 -- Swap
and contract_address in (select contract_address from pool)
),
Expand Down Expand Up @@ -77,6 +79,11 @@ tvl_volume_per_swap as (
on
date_trunc('minute', syncs.evt_block_time) = p1.timestamp
and syncs.token1 = p1.contract_address
where
p0.timestamp >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and p0.timestamp <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
and p1.timestamp >= date_add('day', -1, (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end))
and p1.timestamp <= (case when '{{competitor_end_time}}' = '2100-01-01' then now() else timestamp '{{competitor_end_time}}' end)
)

select
Expand Down
19 changes: 9 additions & 10 deletions cowamm/kpi/competitors/uni_swap_style/largest_uni_style_pools_4303563.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ syncs as (
date_trunc('minute', block_time) as evt_block_time,
varbinary_to_uint256(substr(data, 1, 32)) as reserve0,
varbinary_to_uint256(substr(data, 33, 32)) as reserve1,
rank() over (partition by (logs.contract_address) order by block_time desc, index asc) as latest
rank() over (partition by (logs.contract_address) order by block_time desc, index desc) as latest
from {{blockchain}}.logs
inner join pools
on logs.contract_address = pools.contract_address
Expand All @@ -72,14 +72,13 @@ select distinct
evt_block_time,
reserve0 * p0.price * power(10, -p0.decimals) + reserve1 * p1.price * power(10, -p1.decimals) as tvl
from syncs as s
inner join prices.minute as p0
on
token0 = p0.contract_address
and p0.timestamp = evt_block_time
inner join prices.minute as p1
on
token1 = p1.contract_address
and p1.timestamp = evt_block_time
where latest = 1
inner join prices.day as p0
on token0 = p0.contract_address
inner join prices.day as p1
on token1 = p1.contract_address
where
latest = 1
and p0.timestamp = date_trunc('day', case when ('{{end_time}}' = '2100-01-01' or date('{{end_time}}') = date_trunc('day', now())) then now() - interval '1' day else date('{{end_time}}') end)
and p1.timestamp = date_trunc('day', case when ('{{end_time}}' = '2100-01-01' or date('{{end_time}}') = date_trunc('day', now())) then now() - interval '1' day else date('{{end_time}}') end)
order by tvl desc
limit {{number_of_pools}}
Loading