Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test the effects of a random noise vector in the model #272

Open
dfsnow opened this issue Dec 7, 2024 · 3 comments
Open

Test the effects of a random noise vector in the model #272

dfsnow opened this issue Dec 7, 2024 · 3 comments
Assignees
Labels
method ML technique or method change

Comments

@dfsnow
Copy link
Member

dfsnow commented Dec 7, 2024

Following up on discussions in the office, we should test the perturbation effect of a vector of random noise in the model. Specifically, we should look at feature importance and the stability of estimates.

@dfsnow dfsnow added the method ML technique or method change label Dec 7, 2024
@Damonamajor
Copy link
Contributor

Damonamajor commented Dec 9, 2024

@dfsnow do we want this in something like the equivalent of an EI-issue? Where should it be stored?
The same question for #273

@Damonamajor
Copy link
Contributor

Damonamajor commented Dec 9, 2024

@dfsnow We are also probably going to have a decent number of comparisons / test etc. Do we want to have some set parameters or a baseline to test these things against throughout the modeling season?

@dfsnow
Copy link
Member Author

dfsnow commented Dec 9, 2024

@dfsnow do we want this in something like the equivalent of an EI-issue? Where should it be stored? The same question for #273

For something like this issue, I think it's sufficient to just post findings in the issue comments (including any supporting tables and plots). No need for separate analyses.

@dfsnow We are also probably going to have a decent number of comparisons / test etc. Do we want to have some set parameters or a baseline to test these things against throughout the modeling season?

We'll run a baseline model as soon as #263 is merged. We can use that for comparison for now (until we have 2024 data).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
method ML technique or method change
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants