Enhance debugging

[elie222]

Set a super basic example in my repo:

import { shortest } from "@antiwork/shortest";

shortest("Verify that the user can access the / page");

But would get this each time:

📄 login.test.ts

Test Execution Error: page.waitForLoadState: Timeout 30000ms exceeded.
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

 Tests    0 passed (0)
 Duration   33.35s
 Start at   12:42:56 PM

On a Next.js project. Followed the set up steps.

[m2rads]

@elie222 Thanks for reporting this bug. There are two people that have opened this and this is a bug clearly. I think the issue arises from Playwright waitForLoad method that waits for the pages to be loaded after navigation or initial load of browser.

I am investigating this issue and will open a fix for this soon.

[elie222]

@elie222 Thanks for reporting this bug. There are two people that have opened this and this is a bug clearly. I think the issue arises from Playwright waitForLoad method that waits for the pages to be loaded after navigation or initial load of browser.

I am investigating this issue and will open a fix for this soon.

Thanks. I tried it on https://github.com/elie222/inbox-zero.

Although will take a little time just to get the project running. It is a Next.js app.

I'll take a deeper look myself. If there's a debug / verbose mode may help debug as well.

[elie222]

Ah, I found debug mode:

pnpm shortest --debug-ai

[slavingia]

Maybe should just be --debug or --verbose?

Unless the debugging is specifically only for the AI part

[elie222]

Ya, so --debug-ai didn't do anything to help solve it.
As it didn't get up to the stage of running the AI.

[gladyshcodes]

@elie222 Could you try increasing timeout? Also, what do you see in Chromium (make sure you're not running in headless mode)?

[courthead]

Same issue here. Chromium opens up my site, then nothing happens after that, then timeout after 30 seconds.

[slavingia]

What's the test you're running?

[amk-dev]

I can see some code in browser/manager/index.ts that might cause this
so we're waiting for networkidle event, may be due to poor connection / some background requests that are not resolved within 30s, this code might be a bit troublesome.

const page = await this.context.newPage();
await page.goto(this.normalizeUrl(this.config.baseUrl));
await page.waitForLoadState("networkidle");

i was able to reproduce a similar error, using a sample html page that fires fetches non stop for 30 seconds. ( asked claude to generate, i've attached the code as a gist if anyone wants to reproduce the behaviour )

https://gist.github.com/amk-dev/164614f5b36d2bd2ba0bbf24198df017#file-non-stop-fetches-html

@elie222 @courthead any chance this might be happening with you guys ?

[courthead]

Okay, I got it working. I think it had nothing to do with the tests.

(Here's my only test btw: shortest('Validate the homepage loads and that it displays articles');)

I'm using Ember and I typically serve my app locally by running ember server --port=4200 which makes my app accessible at http://localhost:4200 in my browser. When I do that and then run npx shortest --debug-ai, it opens my site in Chromium and then simply does nothing. It doesn't move the mouse, doesn't click, doesn't scroll, and nothing even gets logged to my Terminal. The output looks like this:

However, another way for me to serve my app locally is to spin up a server with Node/Express, and just have that serve my Ember app, also at http://localhost:4200. When I do it that way and then run npx shortest --debug-ai, shortest works as expected. But I'm not sure what the difference is that allows it to work.

@elie222 @courthead any chance this might be happening with you guys ?

I don't think that's it for me.

[elie222]

@elie222 @courthead any chance this might be happening with you guys ?

Hey, I will check later. Could be that networkidle never happens. Even though it looks like the page has fully loaded in the browser immediately.

[gladyshcodes]

Identifying and resolving bugs like this would be significantly easier with comprehensive logging across the codebase. It’s likely that an error is being thrown but isn’t logged within the catch block.

My proposal is:

• Enhance the Logger service for better usability
• Ensure all errors are logged within catch blocks
• Simplify the --debug-ai flag to just --debug and enable the Logger when this flag is used