Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistency in ARFClassifier's drift_detector and warning_detector Attributes When Saving and Loading #1332

Closed
kdMoura opened this issue Oct 3, 2023 · 4 comments
Labels

Comments

@kdMoura
Copy link

kdMoura commented Oct 3, 2023

Versions

river version: 0.19.0
Python version: 3.9.18
Operating system: Ubuntu 18.04.6 LTS, Ubuntu 22.04.2 LTS

Describe the bug

I have noticed an inconsistency when using the ARFClassifier from the river library in Python. When I create an ARFClassifier object and set its drift_detector and warning_detector attributes to None, I expect them to be None when I load the object from a saved file using pickle. However, loaded_model.drift_detector and loaded_model.warning_detector are not None as I expected. Is there something I'm missing, or is this an issue?

Steps/code to reproduce

from river import forest
import pickle

model = forest.ARFClassifier(n_models=15, seed=33, leaf_prediction="mc", drift_detector=None, warning_detector=None)
# Save the model
with open('model.pkl', 'wb') as f:
    pickle.dump(model, f)

# Load the model
with open('model.pkl', 'rb') as model_file:
    loaded_model = pickle.load(model_file)

# I expect loaded_model.drift_detector and loaded_model.warning_detector to be None, but they're not.
print(loaded_model.drift_detector)
#ADWIN
print(loaded_model.warning_detector)
#ADWIN
@smastelini
Copy link
Member

smastelini commented Oct 3, 2023

Hi @kdMoura, thanks for reporting that.

Does the problem also happen in the development version of River? We recently made some changes in ARF.

Update: as per @MaxHalford comment.

@MaxHalford
Copy link
Member

I don't think this has anything to do with pickle. The attributes are populated when the class is instantiated:

from river import forest

model = forest.ARFClassifier(drift_detector=None, warning_detector=None)
model.drift_detector
ADWIN (
  delta=0.001
  clock=32
  max_buckets=5
  min_window_length=5
  grace_period=10
)

However, isn't this expected behavior? Using None means that default values are used. See the docs:

Screenshot 2023-10-03 at 23 58 12

@smastelini
Copy link
Member

smastelini commented Oct 3, 2023

Actually, this led me to think. We need to revisit the way we set the detectors by default to ensure users can switch off the drift detection part.

I can take care of this.

@smastelini
Copy link
Member

Hey @kdMoura, please check out #1350 :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants