Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace toasts - be more optimistic - prepaid and postpaid #1184

Closed
wants to merge 46 commits into from

Conversation

ekzyis
Copy link
Member

@ekzyis ekzyis commented May 18, 2024

Description

Using undocumented Apollo API for optimistic behavior for zaps, poll votes and bounty payments

Every action which does not need a database id in the client now updates the UI optimistically even if payment is required. This includes (and is limited to) zaps, poll votes and bounty payments. This leads to a not-custodial UX that is equal to the familiar speed of the custodial UX since all payment delays are moved to the background, out of view from the user. When the invoice was paid, the action is submitted to the server as usual with payment hash + hmac.

To support this, the form component now supports optimisticUpdate. When this property is passed a function, its return value will be passed to the following function which calls QueryManager.markOptimisticMutation and cache.removeOptimistic to manually trigger the optimistic cache update and its revert:

export function optimisticUpdate ({ mutation, variables, optimisticResponse, update }) {
  const { cache, queryManager } = getApolloClient()

  const mutationId = String(queryManager.mutationIdCounter)
  queryManager.markMutationOptimistic(optimisticResponse, {
    mutationId,
    document: mutation,
    variables,
    update
  })

  return () => cache.removeOptimistic(mutationId)
}

We call this function before triggering a payment. On payment or submit error, we rollback the optimistic update. The user is only informed in this case via notifications that are stored in window.localStorage.

During the changes, it became clear that the property invoiceable should actually be called prepaid. This makes it purpose more clear.

Optimistic replies and posts via post-paid support

For mutations where the id is required to update the UI in a sane way, the form component now supports a new property: postpaid.

If this property is passed, the form component will check the return value of the submit handler for an invoice. If one was found, it will try to pay it.

This allows us to use the server response to update the UI as fast as if it was paid via the custodial wallet. In the case a external payment is required, the item is inserted as "pending" which makes them only visible to the author and the server response will include an invoice. Since anons are not logged in and thus don't support this optimistic behavior, they don't have access to pending items but need to pay first. Therefore, forms and mutations support prepaid and postpaid simultaneously.

When the invoice is paid, we update the item status to active which makes it visible to everyone.

Based on and thus includes #1071 closes #849 closes #848 supersedes #1161.

TODO

Prepaid

  • optimistic zaps
    • optimism via optimisticUpdate
    • persistence of optimistic update while payment is pending via window.localStorage
      we'll do this after not-custodial zap beta #1178 since we need to be aware of pending stuff on the server anyway if proxy is used
  • optimistic bounty payments
    • optimism via optimisticUpdate
    • persistence of optimistic update while payment is pending via window.localStorage
      we'll do this after not-custodial zap beta #1178 since we need to be aware of pending stuff on the server anyway if proxy is used
  • poll votes
    • optimism via optimisticUpdate
    • persistence of optimistic update while payment is pending via window.localStorage
      we'll do this after not-custodial zap beta #1178 since we need to be aware of pending stuff on the server anyway if proxy is used
  • error notification if payment failed
    related to persistence to some degree since payment should also be considered "failed" if payment + associated action was lost due to page refresh, but we'll skip this case for now as mentioned above
  • error handling if payment failed (reverting persistence)
    see comments above
  • zap undos

Postpaid

  • Add payment.send next to payment.request from Replace useInvoiceable with usePayment #1071
  • Add postpaid to form which checks for returned invoices and tries to pay them
  • insert pending items and return invoice
  • check for payments on server and update item status
  • check for payment failures and update item status
  • update queries to only show pending items to author
  • error notifications
  • on error notification click, navigate to prefilled replies/forms for good retry UX
  • ignore failed items in spam calculation
  • make sure items transition to ACTIVE even if transition query failed after invoice confirmation
  • don't notify users etc. if pending items are created
  • decide what to do with dead links in error notifications
  • update updatedAt manually in raw queries for notification red square to work for some reason, this wasn't necessary
  • prevent replying to or zapping failed items

TODO after review

  • handle action in same tx as confirm_invoice to fix race conditions if autowithdrawals are enabled
  • test edits of pending items:

Editting a pending comment made a modal popup telling me the payment failed. Is that expected?

When a top level comment failed, I clicked on the notification, then submitted the form to retry to the comment, it succeeded but the comment field did not clear

Videos

TBD

Additional Context

  • the idea of using mutationIdCounter and not mutationIdCounter++ as in the library source code was that the mutation will also call an optimistic update with the same id to make sure the UI updates and overwrite the previous layer. If I recall correctly, I found out it actually creates its own layer with its own (automated) revert. Therefore, I considered it not only be redundant but also a possible source of bugs. However, as mentioned, not sure, would need to check this behavior again.

Checklist

Are your changes backwards compatible? Please answer below:

Did you QA this? Could we deploy this straight to production? Please answer below:

For frontend changes: Tested on mobile? Please answer below:

Did you introduce any new environment variables? If so, call them out explicitly here:

@ekzyis ekzyis added enhancement improvements to existing features ux labels May 18, 2024
Copy link
Contributor

coderabbitai bot commented May 18, 2024

Walkthrough

The changes span across multiple files, focusing on enhancing the payment processing capabilities, refining the user experience for pending payments, and optimizing the handling of external payments. Key additions include new hooks for managing invoices and payments, updates to form components to handle query parameters and invoicing, and modifications to various components to support optimistic UI updates for pending payments. The database schema and worker functions have also been updated to support these changes.

Changes

Files/Path Change Summary
components/payment.js Added hooks for managing invoices, WebLN payments, and QR payments.
components/poll-form.js Added query parameters handling, updated upsertPoll mutation, and adjusted initial form values.
components/poll.js Updated imports to use new payment hooks and added toast notifications.
components/qr.js Simplified payment logic.
components/reply.js Added query parameters for pre-filling and expanded GraphQL query for invoices.
components/territory-form.js, components/territory-payment-due.js, pages/rewards/index.js Changed invoiceable prop to prepaid.
components/toast.js, components/toast.module.css Removed unused functions and styles, updated toast handling logic.
components/upvote.js, components/upvote.module.css Updated ItemAct component calls, made handleShortPress async, and added CSS for pending state.
components/use-crossposter.js Removed nested function, imported determineItemType from @/lib/item.
components/webln/index.js, components/webln/nwc.js Removed unused imports, added constants and errors, refactored context value creation.
fragments/notifications.js Added FailedItem fragment with additional fields.
lib/apollo.js Added optimisticUpdate function.
lib/constants.js Added constants JIT_INVOICE_TIMEOUT_MS, DEFAULT_INVOICE_TIMEOUT_MS, and ZAP_UNDO_DELAY_MS.
lib/item.js Introduced determineItemType function.
pages/_app.js Added ClientNotificationProvider before LoggerProvider.
pages/api/lnurlp/[username]/pay.js Modified create_invoice call with additional parameters.
pages/invoices/[id].js Changed Invoice component import to default import.
pages/settings/index.js Added ZAP_UNDO_DELAY_MS to INVOICE_RETENTION_DAYS and updated button display duration.
prisma/migrations/.../migration.sql Added enum types, columns, and functions for handling invoices and items.
worker/action.js, worker/index.js, worker/wallet.js Added functions for handling actions related to invoices and items, updated handling of canceled invoices.
api/resolvers/user.js Added statusClause to various query clauses.

Assessment against linked issues

Objective Addressed Explanation
Refine external pending payment UX (#849) The changes improve the UX for pending payments, but it is unclear if they fully replace toasts as suggested.
Make actions dependent on external payments complete optimistically (#848) The changes support optimistic UI updates for pending payments and store action data on the server.
Handle payment failures and retries (#848) The system now supports handling payment failures and retries through notifications and updated database functions.

Tip

Early Access Features
  • gpt-4o model for chat

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to full the review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@ekzyis ekzyis marked this pull request as draft May 18, 2024 06:55
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Out of diff range and nitpick comments (1)
components/invoice.js (1)

63-67: Handle WebLnNotEnabledError specifically to improve user feedback on payment failures.

Consider enhancing the error message to provide more actionable information to the user.

components/toast.js Show resolved Hide resolved
components/item-act.js Outdated Show resolved Hide resolved
@ekzyis ekzyis force-pushed the wallet-ux-2 branch 2 times, most recently from b6f62a0 to 3988a00 Compare May 18, 2024 11:15
worker/wallet.js Outdated
Comment on lines 369 to 398
await models.$transaction(async (tx) => {
const item = await tx.item.update({
where: { id: itemId },
data: { status: 'ACTIVE' }
})

await tx.user.update({
where: {
id: item.userId
},
data: {
msats: {
decrement: cost - msatsReceived - locked
}
}
})

// run skipped item queries
await tx.itemAct.create({
data: { msats: cost, itemId: item.id, userId: item.userId, act: 'FEE' }
})
if (item.boost > 0) {
await tx.$executeRaw(`SELECT item_act(${item.id}::INTEGER, ${item.userId}::INTEGER, 'BOOST'::"ItemActType", ${item.boost}::INTEGER)`)
}
if (item.maxBid) {
await tx.$executeRaw(`SELECT run_auction(${item.id}::INTEGER);`)
}
}, { isolationLevel: Prisma.TransactionIsolationLevel.Serializable })
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up (I know this is in draft and this might not be what you intend to ship): worst case, this is 5 roundtrips to the db (because it's interactive and not pipelined), and if during that time any of the modified rows are updated by another transaction, this and the other tx will fail.

In general, the longer a serializable tx takes to run, the more likely the failure. Everywhere in the code so far, we only do one roundtrip for serializable txs (and we wrap them all in a retry in serializable)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

worst case, this is 5 roundtrips to the db (because it's interactive and not pipelined) [...] Everywhere in the code so far, we only do one roundtrip for serializable txs (and we wrap them all in a retry in serializable)

Oh, I didn't realize that the serialize calls with an array of queries weren't multiple round trips.

But I agree with you, roundtrips are bad, especially when it's in a serialized tx. I didn't think too much about it here for now, it was just too convenient to use Prisma's API.

But I noticed now that the only reason I use an interactive tx was to fetch the item author. I can do this outside of the tx and then use serialize.

-- here, we immediately deduct as many of the sats that are required for the payment
-- to effectively "lock" them for it. the remainder will be paid via invoice.
-- if the payment fails, we release the locked sats by adding them to the balance again.
UPDATE users SET msats = GREATEST(msats - cost_msats - item.boost, 0) WHERE id = item."userId";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pr is a big lift even without this partial payment change, but you might have this working well enough it's okay to add. But if it is overwhelming getting this extra done, I'd recommend postponing the partial payments until the primary objective of this PR is pristine.

It's usually a lot easier to add extras when the foundation is settled.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhh yes, but so far, I think it's only a matter of adding them back in case of failure which I have to handle anyway. So let's see

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, since deducting user balance is not idempotent and even though LND subscriptions don't trigger on invoice expiration (so idempotency shouldn't be necessary), I removed partial payments in 2c1807d to be sure.

@ekzyis
Copy link
Member Author

ekzyis commented May 21, 2024

Regarding persistence of optimistic updates for prepaid stuff: I noticed we really don't need it. In 8e6f7ba, I (re)implemented client notifications and realized that if something is still pending when we reload the page, we can immediately show it as failed since this means the payment flow was interrupted. So there is never the case of something being pending but the UI not showing this.

<SubmitButton variant={down ? 'danger' : 'success'} className='ms-auto mt-1 px-4' value='TIP'>{down && 'down'}zap</SubmitButton>
</div>
</Form>
<ClientNotifyProvider additionalProps={{ itemId: item.id }}>
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope I'll still like this approach to pass something to notify that the form component can't down the road

Can't tell yet if this is simple clever or confusing clever

@ekzyis ekzyis marked this pull request as ready for review May 23, 2024 11:26
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 18

Outside diff range and nitpick comments (9)
worker/wallet.js (2)

69-70: Clarify the comment regarding invoice notifications.

The comment about SubscribeToInvoices only handling invoice creation and settlement transitions could be expanded to explain the implications more clearly. Consider adding examples or scenarios where this limitation might affect the application's behavior.


247-248: Clarify the handling of withdrawal confirmations.

The handling of the first confirmation for withdrawals could use more detailed logging or comments to explain what happens when code === 0 and why this condition is important.

api/resolvers/notifications.js (1)

154-155: Ensure proper indexing for notification queries.

Given the complexity and potential size of the data involved in the notification queries, ensure that the database is properly indexed on columns used in WHERE clauses to optimize performance.

components/notifications.js (2)

Line range hint 77-95: The NotificationLayout component is well-implemented with appropriate routing logic. Consider adding a comment explaining the purpose of the overrideLinks style application for clarity.

+ // Apply zIndex style to override links when necessary
  style={overrideLinks ? { zIndex: 1 } : undefined}

Line range hint 551-617: The NotificationAlert component is well-crafted, effectively managing notification settings and user preferences. Consider adding detailed comments explaining the logic within the useEffect hooks for better maintainability.

+ // Check support for service worker features and manage notification settings
  useEffect(() => {
    const isSupported = sw.support.serviceWorker && sw.support.pushManager && sw.support.notification
    ...
  }, [sw])
api/resolvers/wallet.js (3)

16-16: Consider enhancing the error message "not ur invoice" to be more professional and informative, such as "You do not have permission to access this invoice."


331-331: Improve the clarity and detail of error messages in the createWithdrawal function to enhance user understanding and troubleshooting.


[!TIP]
Codebase Verification

The cancelInvoice function in api/resolvers/wallet.js does handle HMAC validation securely. The function generates an HMAC using the createHmac function and compares it with the provided HMAC. If they do not match, it throws a GraphQLError with a FORBIDDEN code.

  • The createHmac function is defined in the same file and uses the crypto.createHmac method with a SHA-256 algorithm and a key derived from an environment variable.
  • The comparison is done using a simple equality check, which might be susceptible to timing attacks. However, in api/resolvers/serial.js, a more secure timingSafeEqual method is used for HMAC comparison.

To enhance security, consider using timingSafeEqual for HMAC comparison in the cancelInvoice function as well.

  • Update the HMAC comparison in cancelInvoice to use timingSafeEqual.

Category:

Analysis chain

Line range hint 384-395: Ensure the security and correctness of HMAC generation and comparison in the cancelInvoice function to prevent unauthorized access.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify HMAC security in `cancelInvoice` function.

# Test: Search for HMAC usage in the function. Expect: Secure HMAC generation and comparison.
ast-grep --lang javascript --pattern $'function cancelInvoice($_, $_, $_) {
  $$$
  const hmac2 = createHmac($_)
  if (hmac !== hmac2) {
    $$$
  }
  $$$
}'

Length of output: 158



Script:

#!/bin/bash
# Description: Locate and inspect the `cancelInvoice` function for HMAC handling.

# Step 1: Locate the `cancelInvoice` function definition.
ast-grep --lang javascript --pattern 'function cancelInvoice($_, $_, $_) { $$$ }'

# Step 2: Search for HMAC usage within the located function.
rg 'createHmac' -A 5 -B 5

Length of output: 5558

pages/settings/index.js (1)

1010-1010: Clarify the behavior of the ZAP_UNDO_DELAY_MS constant.

The comment on line 1010 mentions that the undo button is shown for a specific duration based on ZAP_UNDO_DELAY_MS. It would be helpful to add a comment explaining where ZAP_UNDO_DELAY_MS is defined and its typical value to make the code more understandable for new developers or during maintenance.

Comment on lines +57 to +65
let filter = 'AND "Item"."status" <> \'FAILED\''
if (me) {
filter += `AND ("Item"."userId" = ${me.id} OR "Item"."status" <> 'PENDING')`
const [{ item_comments_zaprank_with_me: comments }] = await models.$queryRawUnsafe(
'SELECT item_comments_zaprank_with_me($1::INTEGER, $2::INTEGER, $3::INTEGER, $4::INTEGER, $5, $6)', Number(id), GLOBAL_SEED, Number(me.id), COMMENT_DEPTH_LIMIT, filter, orderBy)
return comments
}

filter += 'AND "Item"."status" <> \'PENDING\''
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optimize the SQL query to prevent potential performance issues with large datasets.

Consider using more efficient SQL constructs or adding appropriate indexes to improve the performance of this query.

Comment on lines +47 to +50
const setItemMeAnonSats = ({ id, amount }) => {
const storageKey = `TIP-item:${id}`
const existingAmount = Number(window.localStorage.getItem(storageKey) || '0')
window.localStorage.setItem(storageKey, existingAmount + amount)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure proper error handling and validation for local storage operations.

const setItemMeAnonSats = ({ id, amount }) => {
  const storageKey = `TIP-item:${id}`
  const existingAmount = Number(window.localStorage.getItem(storageKey) || '0')
+ if (isNaN(existingAmount)) {
+   throw new Error('Invalid amount in local storage');
+ }
  window.localStorage.setItem(storageKey, existingAmount + amount)
}

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
const setItemMeAnonSats = ({ id, amount }) => {
const storageKey = `TIP-item:${id}`
const existingAmount = Number(window.localStorage.getItem(storageKey) || '0')
window.localStorage.setItem(storageKey, existingAmount + amount)
const setItemMeAnonSats = ({ id, amount }) => {
const storageKey = `TIP-item:${id}`
const existingAmount = Number(window.localStorage.getItem(storageKey) || '0')
if (isNaN(existingAmount)) {
throw new Error('Invalid amount in local storage');
}
window.localStorage.setItem(storageKey, existingAmount + amount)
}

@@ -1007,7 +1007,7 @@ const ZapUndosField = () => {
<Info>
<ul className='fw-bold'>
<li>An undo button is shown after every zap that exceeds or is equal to the threshold</li>
<li>The button is shown for 5 seconds</li>
<li>The button is shown for {ZAP_UNDO_DELAY_MS / 1000} seconds</li>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider improving user feedback for the ZapUndosField component.

The ZapUndosField component could benefit from enhanced user feedback. For instance, when the undo button is not shown due to the conditions not being met, displaying a brief message explaining why can help improve user understanding and interaction.

Comment on lines +260 to +270
export function statusClause (me) {
let filter = '"Item"."status" <> \'FAILED\' AND '

if (me) {
filter += `("Item"."userId" = ${me.id} OR "Item"."status" <> 'PENDING')`
} else {
filter += '"Item"."status" <> \'PENDING\''
}

return filter
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor the statusClause function to improve readability and maintainability.

Consider breaking down the function into smaller, more manageable parts or using helper functions to handle different conditions.

Comment on lines +493 to +505
const newFailedItems = await models.item.findFirst({
where: {
userId: me.id,
status: 'FAILED',
updatedAt: {
gt: lastChecked
}
}
})
if (newFailedItems) {
foundNotes()
return true
}
Copy link
Contributor

@coderabbitai coderabbitai bot May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider optimizing the query for checking new failed items.

The current implementation performs a query for each type of notification. This could potentially be optimized by combining some of these queries into a single query, especially if they are checking similar conditions or can be grouped logically.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment reminds me that I probably need to manually update updatedAt in raw queries since it's not based on a trigger.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekzyis, you're absolutely right about the need to manually update the updatedAt field in raw queries, especially since it's not managed by a database trigger in this context. Manually managing this field can ensure that your data remains consistent and accurate, which is crucial for features that rely on the timeliness of data updates.

If you're updating multiple records or have multiple places where this needs to be handled, you might consider abstracting this logic into a helper function or middleware to keep your code DRY and maintainable. Here’s a quick example of how you might implement such a function in JavaScript:

const updateWithTimestamp = async (model, where, data) => {
  const updatedAt = new Date();
  const updatedData = { ...data, updatedAt };
  return await model.update({ where, data: updatedData });
};

This function takes the model, the conditions (where), and the data to be updated, and it automatically appends the current timestamp to the updatedAt field before executing the update.

Let me know if you need further assistance with this or any other part of your implementation!

Copy link
Member Author

@ekzyis ekzyis May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhh, even though I don't see a trigger that does this, updatedAt gets updated after a raw query updates an item 🤔

These are the existing item triggers:

Triggers:
    index_item AFTER INSERT OR UPDATE ON "Item" FOR EACH ROW EXECUTE FUNCTION index_item()
    ncomments_after_comment_trigger AFTER INSERT ON "Item" FOR EACH ROW EXECUTE FUNCTION ncomments_after_comment()
    path_tgr BEFORE INSERT OR UPDATE ON "Item" FOR EACH ROW EXECUTE FUNCTION update_item_path()
    timestamp_item_on_insert AFTER INSERT ON "Item" FOR EACH ROW EXECUTE FUNCTION timestamp_item_on_insert()

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ekzyis, it's interesting to see that the updatedAt field is being updated automatically, even though there isn't a specific trigger listed for this action. This behavior might be due to a couple of reasons:

  1. Database Abstraction or ORM Behavior: If the project uses an ORM (Object-Relational Mapping) or any database abstraction layer, it's common for these tools to manage updatedAt fields automatically. They often do this during any update operation without needing explicit triggers in the database.

  2. Existing Trigger Handling More Than Its Name Suggests: The trigger timestamp_item_on_insert might be slightly misleading by its name. It's possible that this trigger is configured to handle updates as well as inserts, despite what the name suggests. This would allow it to update the updatedAt field on any modification to the row.

If you need to confirm or inspect this behavior further, you might consider checking the definition of the timestamp_item_on_insert trigger to see if it indeed handles updates. Alternatively, reviewing the ORM or database layer's configuration or documentation could provide insights into this automatic handling of the updatedAt field.

Let me know if you need more detailed information on this or any other queries!

@@ -327,7 +328,7 @@ export default {
}
},
Mutation: {
createInvoice: async (parent, { amount, hodlInvoice = false, expireSecs = 3600 }, { me, models, lnd, headers }) => {
createInvoice: async (parent, { amount, hodlInvoice = false, expireSecs = (DEFAULT_INVOICE_TIMEOUT_MS / 1000) }, { me, models, lnd, headers }) => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review and optimize transaction management in the upsertWallet function to ensure data consistency and error resilience.

Comment on lines +25 to +31
if (data) {
invoice = data.invoice
}

if (error) {
return <div>{error.toString()}</div>
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good error handling in the component. Displaying the error message directly can help in debugging, but consider sanitizing or formatting the error for end users.

Comment on lines 145 to 181
const onClickFailedItem = ({ n }) => {
const { parentId, title, url, text, subName: sub, boost, bounty } = n.item
const isPost = !!n.item.title

if (isPost) {
const type = determineItemType(n.item)
const query = { type, title, text, sub }
if (boost > 0) {
query.boost = boost
}
if (type === 'link') {
query.url = url
}
if (type === 'bounty' && bounty > 0) {
query.bounty = bounty
}
return {
href: {
pathname: '/post',
query
}
}
}

const rootId = commentSubTreeRootId(n.item)
const query = { id: rootId, text }
if (Number(rootId) !== Number(parentId)) {
query.commentId = parentId
}
return {
href: {
pathname: '/items/[id]',
query
},
as: `/items/${rootId}`
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The onClickFailedItem function correctly handles link generation for failed items. Consider adding error handling or a default case for unexpected item types to enhance robustness.

+ if (!['link', 'bounty', 'post', 'comment'].includes(type)) {
+   console.error('Unexpected item type:', type);
+   return {}; // Return empty to prevent navigation
+ }

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
const onClickFailedItem = ({ n }) => {
const { parentId, title, url, text, subName: sub, boost, bounty } = n.item
const isPost = !!n.item.title
if (isPost) {
const type = determineItemType(n.item)
const query = { type, title, text, sub }
if (boost > 0) {
query.boost = boost
}
if (type === 'link') {
query.url = url
}
if (type === 'bounty' && bounty > 0) {
query.bounty = bounty
}
return {
href: {
pathname: '/post',
query
}
}
}
const rootId = commentSubTreeRootId(n.item)
const query = { id: rootId, text }
if (Number(rootId) !== Number(parentId)) {
query.commentId = parentId
}
return {
href: {
pathname: '/items/[id]',
query
},
as: `/items/${rootId}`
}
}
const onClickFailedItem = ({ n }) => {
const { parentId, title, url, text, subName: sub, boost, bounty } = n.item
const isPost = !!n.item.title
if (isPost) {
const type = determineItemType(n.item)
const query = { type, title, text, sub }
if (boost > 0) {
query.boost = boost
}
if (type === 'link') {
query.url = url
}
if (type === 'bounty' && bounty > 0) {
query.bounty = bounty
}
if (!['link', 'bounty', 'post', 'comment'].includes(type)) {
console.error('Unexpected item type:', type);
return {}; // Return empty to prevent navigation
}
return {
href: {
pathname: '/post',
query
}
}
}
const rootId = commentSubTreeRootId(n.item)
const query = { id: rootId, text }
if (Number(rootId) !== Number(parentId)) {
query.commentId = parentId
}
return {
href: {
pathname: '/items/[id]',
query
},
as: `/items/${rootId}`
}
}

Comment on lines +529 to +550
function FailedItem ({ n }) {
const isPost = !!n.item.title
const type = isPost ? determineItemType(n.item) : undefined
const message = `failed to ${type ? `create ${type}` : 'reply'}: invoice was not paid`
return (
<div className='ms-2'>
<small className='fw-bold text-danger'>
{n.reason ? `${message}: ${n.reason}` : message}
<small className='text-muted ms-1 fw-normal' suppressHydrationWarning>{timeSince(new Date(n.sortTime))}</small>
</small>
{isPost
? <Item item={n.item} />
: (
<div className='pb-2'>
<RootProvider root={n.item.root}>
<Comment item={n.item} noReply includeParent noComments clickToContext />
</RootProvider>
</div>
)}
</div>
)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FailedItem component effectively communicates failure details to the user. Consider using template literals for constructing the failure message to improve readability.

- const message = `failed to ${type ? `create ${type}` : 'reply'}: invoice was not paid`
+ const action = type ? `create ${type}` : 'reply';
+ const message = `failed to ${action}: invoice was not paid`;

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
function FailedItem ({ n }) {
const isPost = !!n.item.title
const type = isPost ? determineItemType(n.item) : undefined
const message = `failed to ${type ? `create ${type}` : 'reply'}: invoice was not paid`
return (
<div className='ms-2'>
<small className='fw-bold text-danger'>
{n.reason ? `${message}: ${n.reason}` : message}
<small className='text-muted ms-1 fw-normal' suppressHydrationWarning>{timeSince(new Date(n.sortTime))}</small>
</small>
{isPost
? <Item item={n.item} />
: (
<div className='pb-2'>
<RootProvider root={n.item.root}>
<Comment item={n.item} noReply includeParent noComments clickToContext />
</RootProvider>
</div>
)}
</div>
)
}
function FailedItem ({ n }) {
const isPost = !!n.item.title
const type = isPost ? determineItemType(n.item) : undefined
const action = type ? `create ${type}` : 'reply';
const message = `failed to ${action}: invoice was not paid`;
return (
<div className='ms-2'>
<small className='fw-bold text-danger'>
{n.reason ? `${message}: ${n.reason}` : message}
<small className='text-muted ms-1 fw-normal' suppressHydrationWarning>{timeSince(new Date(n.sortTime))}</small>
</small>
{isPost
? <Item item={n.item} />
: (
<div className='pb-2'>
<RootProvider root={n.item.root}>
<Comment item={n.item} noReply includeParent noComments clickToContext />
</RootProvider>
</div>
)}
</div>
)
}

Comment on lines +83 to +90
invoice {
id
bolt11
hash
hmac
expiresAt
satsRequested
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tip

Codebase Verification

Ensure alignment of invoice object structure with backend schema.

The invoice object structure in the upsertLink mutation should match the backend schema definitions. Here are the relevant structures:

  • Frontend (components/link-form.js):

    invoice {
      id
      bolt11
      hash
      hmac
      expiresAt
      satsRequested
    }
  • Backend (prisma/schema.prisma):

    model Invoice {
      id             Int       @id @default(autoincrement())
      createdAt      DateTime  @default(now()) @map("created_at")
      updatedAt      DateTime  @default(now()) @updatedAt @map("updated_at")
      userId         Int
      hash           String    @unique(map: "Invoice.hash_unique")
      preimage       String?   @unique(map: "Invoice.preimage_unique")
      isHeld         Boolean?
      bolt11         String
      expiresAt      DateTime
      confirmedAt    DateTime?
    }
  • Backend (api/typeDefs/wallet.js):

    type Invoice {
      id: ID!
      createdAt: Date!
      hash: String!
      bolt11: String!
      expiresAt: Date!
      cancelled: Boolean!
      confirmedAt: Date
      satsReceived: Int
      satsRequested: Int!
      nostr: JSONObject
    }

The invoice object in the frontend is missing some fields present in the backend schema. Ensure that the frontend structure includes all necessary fields expected by the backend.

  • Add missing fields such as createdAt, confirmedAt, cancelled, satsReceived, and nostr if they are required by the backend.
Analysis chain

Ensure that the invoice object structure in the upsertLink mutation aligns with backend expectations.

Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify the `invoice` object structure in the `upsertLink` mutation.

# Test: Search for the backend schema for `invoice`. Expect: Alignment with the frontend structure.
rg --type js "invoice" api/schema.js

Length of output: 90



Script:

#!/bin/bash
# Description: Locate the backend schema file by searching for common schema-related keywords.

# Search for files containing the keyword 'schema' which might indicate the backend schema.
rg --files-with-matches "schema"

# Search for files containing the keyword 'invoice' to locate the relevant schema definition.
rg --files-with-matches "invoice"

Length of output: 3570



Script:

#!/bin/bash
# Description: Verify the `invoice` object structure in the backend schema files.

# Search for the `invoice` object definition in `prisma/schema.prisma`.
rg "invoice" prisma/schema.prisma

# Search for the `invoice` object definition in `api/typeDefs/wallet.js`.
rg "invoice" api/typeDefs/wallet.js

Length of output: 262



Script:

#!/bin/bash
# Description: Extract the full definition of the `Invoice` object from the backend schema files.

# Extract the full definition of the `Invoice` object from `prisma/schema.prisma`.
rg -A 10 "model Invoice" prisma/schema.prisma

# Extract the full definition of the `Invoice` object from `api/typeDefs/wallet.js`.
rg -A 10 "type Invoice" api/typeDefs/wallet.js

Length of output: 781

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (3)
api/resolvers/user.js (3)

Line range hint 2-3: Update import syntax for Node.js built-in modules.

- import { readFile } from 'fs/promises'
- import { join, resolve } from 'path'
+ import { readFile } from 'node:fs/promises'
+ import { join, resolve } from 'node:path'

This change ensures compatibility with the latest Node.js module resolution enhancements, which recommend specifying the node: protocol when importing built-in modules.


Line range hint 21-24: Consider using for...of instead of forEach for better error handling and async support.

- fileContent.split('\n')
-   .map(line => line.trim())
-   .filter(line => !!line)
-   .forEach(name => set.add(name))
+ for (const line of fileContent.split('\n').map(line => line.trim()).filter(line => !!line)) {
+   set.add(line);
+ }

Using for...of allows for easier integration of asynchronous operations and better handling of exceptions within loops.


Line range hint 628-630: Remove unnecessary else clause after early return.

- } else {
-   return await models.user.update({ where: { id: me.id }, data: { ...data, nostrRelays: { deleteMany: {} } } })
- }
+ return await models.user.update({ where: { id: me.id }, data: { ...data, nostrRelays: { deleteMany: {} } } })

Since the previous branches in the conditional structure use return, the else clause is redundant and can be omitted for cleaner code.

@huumn
Copy link
Member

huumn commented May 23, 2024

The zapping experience seems to work really well. I didn't notice an issue there yet or encounter an error. It appears to just work.

Things noticed after a few minutes of QA:

  1. I boosted a post and it's stuck as PENDING
  2. When a top level comment failed, I clicked on the notification, then submitted the form to retry to the comment, it succeeded but the comment field did not clear
  3. Editting a pending comment made a modal popup telling me the payment failed. Is that expected?
  4. When I have an autowithdrawal method attached, paying for things races with autowithdraw (after I paid for the post in (1), it autowithdrew 51 sats)

(2, 3, 4) aren't really regressions or "big deals" but (1) is pretty nasty as it looks to the person that created it that it succeeded but it didn't and it's unclear what they should do now. I'd guess there's a problem in the failure finalize code but I don't know. It might be easier to recreate if you add an autowithdraw method.

That's after just a few minutes. There's a lot going on that's changed in this code, so it's hard for me to know what is causing what.

I'll rebase #1178 on this and begin mending them/refactoring together in the meantime. I'll let you know if anything comes up.

Edit: it looks like the payment in (1) was confirmed but the Item is still pending.

stackernews=# select * from "Invoice" where "actionId" = 459394;
   id   |       created_at        |       updated_at        | userId |                               hash                               |                        
                                                                                                                              bolt11                             
                                                                                                                          |       expiresAt        |       confir
medAt       | cancelled | msatsRequested | msatsReceived |             desc              | preimage | isHeld | comment | lud18Data | confirmedIndex | actionId | 
actionType |           actionData            
--------+-------------------------+-------------------------+--------+------------------------------------------------------------------+------------------------
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------+------------------------+-------------
------------+-----------+----------------+---------------+-------------------------------+----------+--------+---------+-----------+----------------+----------+-
-----------+---------------------------------
 159722 | 2024-05-23 19:38:43.926 | 2024-05-23 19:38:57.794 |    616 | 1918ea20fa668e98a6b87943d168369b3334f2164c0bf01aa35e8d4a523520b4 | lnbcrt510n1pnylx6rpp5ry
vw5g86v68f3f4c09paz6pknvenfuskfs9lqx4rt6x55534yz6qdp0gdex2ct5d9hxwgrfw3jk6gr0dcs8xarpvd4k2u3wdejhwuccqzzsxqzftsp595gdhx2smesau4l3ea4sq7muvf5725pugdfr09fsk8ymnfzf
azhq9qyyssqc6jrd4e50t54gkzf3hqw2v28xkqz8vu25gp35cq4at5x3t0n4y69hdpc348rgppqfkyvl9tr73nftqrl8h5jp6qzarxxgnqq6ul902cp3sq0lp | 2024-05-23 19:43:43.91 | 2024-05-23 1
9:38:57.599 | f         |          51000 |         51000 | Creating item on stacker.news |          |        |         |           |             43 |   459394 | 
ITEM       | {"cost": 51000, "credits": 510}
(1 row)

stackernews=# select * from "Item" where status <> 'ACTIVE';
   id   |       created_at        |       updated_at        |   title   |              text               | url | userId | parentId |     path      | pinId | lat
itude | location | longitude | maxBid | maxSalary | minSalary | remote | subName | statusUpdatedAt | status  | company | weightedVotes | boost | uploadId | pollC
ost | paidImgLink | commentMsats | lastCommentAt | ncomments | msats | weightedDownVotes | bio | freebie | deletedAt | otsFile | otsHash | bounty | rootId | boun
tyPaidTo | upvotes | weightedComments | imgproxyUrls | noteId | outlawed | pollExpiresAt | lastZapAt 
--------+-------------------------+-------------------------+-----------+---------------------------------+-----+--------+----------+---------------+-------+----
------+----------+-----------+--------+-----------+-----------+--------+---------+-----------------+---------+---------+---------------+-------+----------+------
----+-------------+--------------+---------------+-----------+-------+-------------------+-----+---------+-----------+---------+---------+--------+--------+-----
---------+---------+------------------+--------------+--------+----------+---------------+-----------
 459389 | 2024-05-23 19:29:39.594 | 2024-05-23 19:29:51.687 |           | fdasjldjlajlfdasfdsaeaefwa33333 |     |    616 |   458433 | 458433.459389 |       |    
      |          |           |        |           |           |        |         |                 | FAILED  |         |             0 |     0 |          |      
    | f           |            0 |               |         0 |     0 |                 0 | f   | f       |           |         |         |        | 458433 |     
         |       0 |                0 | {}           |        | f        |               | 
 459394 | 2024-05-23 19:38:43.926 | 2024-05-23 19:38:49.129 | reqwwfasd | fdsafdsafdsa                    |     |    616 |          | 459394        |       |    
      |          |           |        |           |           |        | bitcoin |                 | PENDING |         |             0 | 50000 |          |      
    | f           |            0 |               |         0 |     0 |                 0 | f   | f       |           |         |         |        |        |     
         |       0 |                0 | {}           |        | f        |               | 
(2 rows)

JSON.stringify({ ...item, status: 'PENDING' }), JSON.stringify(fwdUsers), JSON.stringify(options), uploadIds))

// set required invoice data to update item on payment
const actionData = { cost: err.cost, credits: err.balance };
Copy link
Member

@huumn huumn May 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If these fields need to be saved, might we do better storing them as columns with types and naming them in the db layer?

@huumn
Copy link
Member

huumn commented May 23, 2024

Thinking through my own changes in #1178 alongside this makes me think we are going to need a refactor to make our planned changes robust.

If we solve the problem of (1) in a robust way and we can reason through the robustness of all the states of payments<->actions in this pr, we can ship this before the refactor. Bugs like this are evidence the code has become too much to handle.

@ekzyis
Copy link
Member Author

ekzyis commented May 23, 2024

  1. When a top level comment failed, I clicked on the notification, then submitted the form to retry to the comment, it succeeded but the comment field did not clear

Sounds like a missing resetForm call, should be easy to fix

  1. Editting a pending comment made a modal popup telling me the payment failed. Is that expected?

No 👀 I actually didn't test this, since I didn't change how items are updated and edits should have no cost associated to them so it shouldn't trigger any payment code. But will test now

  1. I boosted a post and it's stuck as PENDING
  1. When I have an autowithdrawal method attached, paying for things races with autowithdraw (after I paid for the post in (1), it autowithdrew 51 sats)

If we solve the problem of (1) in a robust way and we can reason through the robustness of all the states of payments<->actions in this pr, we can ship this before the refactor.

One thing I wanted to do (and I think you also mentioned this) was to run the action in the same tx as the invoice gets confirmed. Currently, it's not since the state transition is not idempotent due to item side effects so I used the return value of confirm_invoice to see if it's the first confirmation.

Running both actions in the same tx should fix the race condition with autowithdrawals. I could do this by creating a postgres function that runs the code in handleAction since then I can rely on serialized transactions to check if it's the first confirmation (just like confirm_invoice).

We don't run into this problem with prepaid because the hash+hmac approach does that: creating and thus paying for the item is in the same tx as confirming the invoice.

Edit: it looks like the payment in (1) was confirmed but the Item is still pending.

There is a worker job which runs finalizeAction after the invoice is expired. It's there to make sure all state transitions are guaranteed to happen eventually (when the invoice expires). However, if the state transition to ACTIVE also fails there, it will be pending forever. That's a bug, I should just mark the item as FAILED then.

So I think the most important remaining TODO here is to create handle_action as a postgres function that we can run in the same tx as confirm_invoice. This should fix conflicts with autowithdrawals and thus 1) and 4) (and possibly other payment problems)

@huumn
Copy link
Member

huumn commented May 23, 2024

FYI any of the code here might end up getting completely rewritten in the refactor and we need to do the refactor soon. All of our payment stuff was complicated before these changes we are making, and now it's even more so (ie there are three ways to pay for something now vs two). The code on the frontend got a lot better, but the backend code got a lot worse.

Just wanted to caution against getting too attached to any particular bug fix when we really need a systemic fix. Still, happy to ship this once it's deterministic and bug-free.

For a hint of how complicated this is, here's the abstraction requirements table from my notes:

anonable optimistic qr payable p2p wrapped side effects fee credits payable
posts x x x x x
comments x x x x x
zaps x x x x x x
downzaps x x x x
poll votes x x x
territory purchases x x
donations x x x x
update posts x? x x x
update comments x? x x x

@ekzyis ekzyis marked this pull request as draft May 24, 2024 18:49
@ekzyis
Copy link
Member Author

ekzyis commented May 27, 2024

I realized that fixing edits of pending items would also require changes to update_item (a similar flow as with create_item essentially) that will be overwritten by the systemic fix we discussed.

However, I think even the systemic fix we discussed wouldn't handle edits properly. Edits of pending items means that we now potentially have two invoices that could be paid separately but they require payment in order since it doesn't make sense to pay for an edit when the item itself wasn't paid yet and thus could still fail. Additionally, edits are the only actions that we have that mutate data instead of creating a row that we could mark as pending.

I see a few ways to approach this problem:

  1. edits of pending items aren't allowed. you must first pay for the item before you can edit it. an edit that wasn't paid yet marks the item as pending again.
  2. items never fail but can always be paid later via a new invoice
  3. edits cancel the existing invoice for the pending item and create a new invoice that pays for the item + edit

(or a combination of them; 2) is something we discussed already iirc)

However, as mentioned, I consider this to be better done in the systemic fix instead of me messing with the backend more in this PR to get pending items working without bugs.

So I decided to indeed create a PR with only the frontend changes. I think we should merge them even if we'll also move zaps etc. to the server as pending acts since they include important changes to cleanup frontend code (removal of toast flows). They also already improve UX even if there are only few stackers who use attached wallets like me.

@ekzyis
Copy link
Member Author

ekzyis commented May 28, 2024

Superseded by #1194 and #1195

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement improvements to existing features ux
Projects
None yet
Development

Successfully merging this pull request may close these issues.

External pending payment UX Make actions dependent on external payments complete optimistically
2 participants