Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: [enforce fail at pybind_state.h:425] . Exception encountered running PythonOp function: ValueError: min() arg is an empty sequence #14

Closed
yuyijie1995 opened this issue Dec 1, 2019 · 2 comments

Comments

@yuyijie1995
Copy link

I download the pretrained model from "'https://dl.fbaipublicfiles.com/detectron'" and put it in the /tmp/detectron/ImageNetPretrained/25093814
BUT the error happend when I start to train the model. I do not understand this error meaning.
Traceback (most recent call last):
File "/home/wrc/CBNet/tools/train_net.py", line 132, in
main()
File "/home/wrc/CBNet/tools/train_net.py", line 114, in main
checkpoints = detectron.utils.train.train_model()
File "/home/wrc/CBNet/detectron/utils/train.py", line 67, in train_model
workspace.RunNet(model.net.Proto().name)
File "/home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/workspace.py", line 250, in RunNet
StringifyNetName(name), num_iter, allow_fail,
File "/home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/workspace.py", line 211, in CallWithExceptionIntercept
return func(*args, **kwargs)
RuntimeError: [enforce fail at pybind_state.h:425] . Exception encountered running PythonOp function: ValueError: min() arg is an empty sequence

At:
/home/wrc/CBNet/detectron/utils/segms.py(136): polys_to_boxes
/home/wrc/CBNet/detectron/roi_data/mask_rcnn.py(46): add_mask_rcnn_blobs
/home/wrc/CBNet/detectron/roi_data/cascade_rcnn.py(193): _sample_rois
/home/wrc/CBNet/detectron/roi_data/cascade_rcnn.py(105): add_cascade_rcnn_blobs
/home/wrc/CBNet/detectron/ops/distribute_cascade_proposals.py(61): forward

Error from operator:
input: "gpu_0/proposals_3" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois_3" output: "gpu_0/labels_int32_3" output: "gpu_0/bbox_targets_3" output: "gpu_0/bbox_inside_weights_3" output: "gpu_0/bbox_outside_weights_3" output: "gpu_0/mapped_gt_boxes_3" output: "gpu_0/mask_rois" output: "gpu_0/roi_has_mask_int32" output: "gpu_0/masks_int32" output: "gpu_0/rois_3_fpn2" output: "gpu_0/rois_3_fpn3" output: "gpu_0/rois_3_fpn4" output: "gpu_0/rois_3_fpn5" output: "gpu_0/rois_3_idx_restore_int32" output: "gpu_0/mask_rois_fpn2" output: "gpu_0/mask_rois_fpn3" output: "gpu_0/mask_rois_fpn4" output: "gpu_0/mask_rois_fpn5" output: "gpu_0/mask_rois_idx_restore_int32" name: "DistributeCascadeProposalsOp:gpu_0/proposals_3,gpu_0/roidb,gpu_0/im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:11" } arg { name: "grad_output_indices" } device_option { device_type: 0 }Error from operator:
input: "gpu_0/proposals_3" input: "gpu_0/roidb" input: "gpu_0/im_info" output: "gpu_0/rois_3" output: "gpu_0/labels_int32_3" output: "gpu_0/bbox_targets_3" output: "gpu_0/bbox_inside_weights_3" output: "gpu_0/bbox_outside_weights_3" output: "gpu_0/mapped_gt_boxes_3" output: "gpu_0/mask_rois" output: "gpu_0/roi_has_mask_int32" output: "gpu_0/masks_int32" output: "gpu_0/rois_3_fpn2" output: "gpu_0/rois_3_fpn3" output: "gpu_0/rois_3_fpn4" output: "gpu_0/rois_3_fpn5" output: "gpu_0/rois_3_idx_restore_int32" output: "gpu_0/mask_rois_fpn2" output: "gpu_0/mask_rois_fpn3" output: "gpu_0/mask_rois_fpn4" output: "gpu_0/mask_rois_fpn5" output: "gpu_0/mask_rois_idx_restore_int32" name: "DistributeCascadeProposalsOp:gpu_0/proposals_3,gpu_0/roidb,gpu_0/im_info" type: "Python" arg { name: "grad_input_indices" } arg { name: "token" s: "forward:11" } arg { name: "grad_output_indices" } device_option { device_type: 1 device_id: 0 }frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x59 (0x7f2dc790c409 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #1: + 0xa2b85 (0x7f2dc805ab85 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #2: + 0xa0fe7 (0x7f2dc8058fe7 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #3: + 0xea931 (0x7f2dc80a2931 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #4: + 0xe8ffd (0x7f2dc80a0ffd in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/caffe2_pybind11_state_gpu.so)
frame #5: caffe2::AsyncNetBase::run(int, int) + 0x144 (0x7f2da8e92b94 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #6: + 0x168f009 (0x7f2da8e99009 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libcaffe2.so)
frame #7: c10::ThreadPool::main_loop(unsigned long) + 0x2a3 (0x7f2dc79062f3 in /home/wrc/anaconda3/envs/py27/lib/python2.7/site-packages/caffe2/python/../../torch/lib/libc10.so)
frame #8: + 0xc8421 (0x7f2ddc0a1421 in /home/wrc/anaconda3/envs/py27/bin/../lib/libstdc++.so.6)
frame #9: + 0x76ba (0x7f2de87036ba in /lib/x86_64-linux-gnu/libpthread.so.0)
frame #10: clone + 0x6d (0x7f2de7d2941d in /lib/x86_64-linux-gnu/libc.so.6)

@mzeimet
Copy link

mzeimet commented Dec 4, 2019

if i remember correctly that error occurs when your dataset is incorrect. been a while since i solved this error for me though, so no guarantee. do you have masks in your dataset?

@yuyijie1995
Copy link
Author

I added the mask and it works! thanks a lot.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants