-
Notifications
You must be signed in to change notification settings - Fork 18
/
Copy pathChangeLog
executable file
·1004 lines (882 loc) · 38 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
CHANGES
=======
4.25.0
------
* release(4.25.0): zarr support, downsample method selection
* install: bump cloudvolume version
* feat: zarr volumes should display x,y,z,c,t instead of t,c,z,y,x
* fix: incorrect parameter for downsampling
* feat: igneous view supports more than precomputed
* install: ensure forward-compatibility with future cloud-volumes
* feat: add downsample method support to image xfer
* fix(cli): show threshold-gte and theshold-lte as integers in --help
* refactor: remove cloudvolume view import
* fix: wrong variable name in shard downsample
4.24.0
------
* release(4.24.0): some fixes, adds min and max pooling to downsampling
* feat: support min and max pooling for downsampling (#176)
* Add missing task creation imports to skeleton.py (#174)
* fix: account for num\_channels in downsample mip calculation
4.23.0
------
* release(4.23.0): support for cross section analysis on skels
* docs: mention ptq tool
* feat: for small numbers of skel labels, estimate bounding box from meshes
* redesign: remove support for pickled frag files
* fix: clamp huge\_bbox
* fix: don't require a queue for skeleton forging
* redesign: change memory target conflicting with num mips to a warning
* fix: make error message an f-string
* fix: don't expand zero-volume sharded downsamples
* perf: reduce number of external requests with crackle encoded cache
* fix: target shape centroid and then cluster bad points
* fix: floating point error downstream of center calculation
* fix: improve handling of cross sectional area contacts
* feat: allow skeletons to have integer attributes by setting flag
* feat: add stop\_layer to TransferTask
* feat: support stop\_layer for ImageShardTransferTask
* chore: update changelog
* refactor: add type annotations to SkeletonTask
* docs: Fix Typo in README (#170)
* docs: Update contrast eq command in README (#171)
4.22.1
------
* release(4.22.1): fixes downsampling issue
* fix: really fix num\_mips issue
* fix(downsample): check num\_mips is not none before comparison
* test: update tests to 3.12, remove 3.7
* feat: support fill\_holes feature for cross section estimation
* fix: vastly reduce cross section underestimates
* fix: handle numpy files with a '.' in the name more gracefully
* fix: add help message to unsupported formats for igneous image create
* refactor: PrecomputedSkeleton -> Skeleton
* feat: remove queue requirement for skeleton merging
* fix: don't duplicate cross\_sectional\_area attr in info file
* fix(cli): show default for fix branching
* feat(cli): add -n to execute a certain number of tasks
4.22.0
------
* release(4.22.0): adds --cross-section to skeleton forging
* feat(SkeletonTask): compute cross sectional area (#166)
* docs: add Ryan to authors (thank you!)
4.21.0
------
* chore: update setup.cfg
* feat: support --xrange,--yrange,--zrange in (#165)
* feat(downsample): use memory limit shape when doing limited downsamples
* feat: add --labels to skeleton forging
* fix: downsampling should not affect the current mips encoding
* fix: exclude png in another location from adding gzip encoding
* fix: don't allow gzip/br on top of jpeg or png encoding
4.20.3
------
* release(4.20.3): fixes contrast normalization CLI
* fix: crashing bug in CLI for contrast normalization
4.20.2
------
* release(4.20.2): fixes bug in downsample
* fix: update info file
* fix: error in refactoring
* feat: add encoding\_level to downsample
4.20.1
------
* release(4.20.1): fixes type of num\_mips
* fix: num\_mips should be interpreted as an int
4.20.0
------
* fix: no reference to sharding in the unsharded function
* feat: add memory\_target to unsharded downsample
* chore: update authors
4.19.3
------
* release(4.19.3): float res, performance enhancements, fix shard imgs
* fix: accomodate offset volumes for shard transfer
* fix: skeletonization when resolution is non-integer value (#158)
* fix: pass cache parameter to ShardedSkeletonMergeTask
* Reduce read requests when creating sharded skeletons and meshes (#156)
* fix: remove debugging statement
* perf: add direct transfer feature to shard transfer
* fix: default value for bounds.minpt
* fix: using the right variable name for shards
* fix(image/xfer): cutout for unsharded
* fix: shards do not recognize voxel offsets so bounds should be expanded
* fix: handling offsets and dest\_voxel\_offset better for shards
4.19.2
------
* fix: processing of mip levels in shard cutout xfer
4.19.1
------
* release(4.19.1): fix igneous create
* fix(cli/create): broken extension normalization
4.19.0
------
* release(4.19.0): updated image create
* feat(cli/create): support for npy, hdf5, and crackle files
* feat: load neuroglancer remotes in RGB mode
* fix: missing urllib import
* feat(create): support np.memmap better
* chore: remove outdated comments
* feat: igneous view takes one fewer click + automatic RGB rendering
* fix: memory target did not accomodate multiple channels
* refactor: whitespace in create downsampling tasks
4.18.0
------
* release(4.18.0): new igneous image create command
* feat: add igneous image create method for npy files
4.17.0
------
* release(4.17.0): can set minishard index encoding for image shard transfer
* fix: unlock compatible with no info file
* feat: allow setting minishard index encoding
* docs: update formatting of instructions
4.16.0
------
* release(4.16.0): features and bugfixes
* docs: "Tissiue ROI" to be more specific
* docs: discuss how to use the ROI computation
* feat: roi detection (#151)
* fix: make ccl auto mode compatible with LocalTaskQueue
* fix: don't retain sharded property when doing a cutout
* fix: missing import
* feat: use in-memory queue if no queue specified
* refactor: separate network call
* test: python 3.11
* perf: switch out compresso for crackle in ccl
* test: ccl connectomics (#150)
* test: add connectomics volume test for ccl
* chore: update supported python versions
* fix: compress option was inoperative for voxel sum
* docs: remove reference to a database
* fix: use layerpath instead of cloudpath
* docs: better formatting of clean command
* feat: remove temp files for unsharded
* chore: whitespace
* feat: add sharded cleaning function for meshes and skeletons
4.15.2
------
* release(4.15.2): Fixes unbound local error
* feat: reorder task (#149)
* fix unbound local error in image xfer --cutout (#148)
* fix: 3D downsampling for sharded images
4.15.0
------
* release(4.15.0): --compress in voxel sum, better caching of voxel counts
* perf: cache voxel counts file as compressed zstd in memory
* feat: add compression option to voxel counts
4.14.0
------
* release(4.14.0): voxel ct more efficient, fix sharded ds, fix encoding
* fix: match original encoding by default during xfer
* fix: downsample encoding should match previous encoding by default
* fix(downsample): border effect for sharded downsample
* fix(downsample): sharded downsample fails for some non-zero offsets
* fixtest: voxel counting test working
* fix: update content-type for intmap
* perf: switch to using IntMap and batch JSON for computing voxel sums
* fix: needed to multiply not divide for lower mips
* fix: adjust bounds iteration when cutout is specified
4.13.0
------
* release(4.13.0): many fixes and improvements since 4.12
* fix: add cutout feature to sharded image transfers
* fix: ensure mesh info doesn't get overwritten
* fix: consolidate was added to the wrong position
* perf: download voxel counts if remote
* feat: add progress and don't check crc for counts
* feat: add bounds and "cutout" option to image xfer
* feat: use global voxel counts for dust threshold (#146)
* feat: compute global voxel counts per label (#145)
* fix: need to call consolidate during sharded skel merge
* docs: update blurb about sharded on CLI
* feat: allow disabling data\_encoding=gzip for shards
4.12.0
------
* feat+perf: adds max\_labels\_per\_shard to skeleton shard production
* fix: missing parameter, adjust docs
* refactor(image): remove unused "CloudVolumePrecomputed" import
* docs: -x is now reliable for sqs
* docs: remove word "beta" from command line
* docs: update citation file to point to paper
* docs: fix link to paper
* docs: add video tutorial link
* docs: add paper badge
* fix(ccl): clean dest info for relabel
4.11.1
------
* release(4.11.1): allow --memory to work for image xfer
* fix(cli/xfer): stop overriding --memory for xfer command
4.11.0
------
* release(4.11.0): improvements and fixes to multires mesh transfers
* redesign(mesh/multires): make default nlod = 0 instead of 1
* redesign(mesh/multires): reduce min\_chunk\_size to 256 from 512
* redesign(mesh/multires): incr. aggressiveness from 5 to 5.5
* fix(multires/mesh): format was corrupted by degenerate objects
* feat(cli): add nlod to mesh xfer
* fix(mesh xfer): better degenerate detection, check zero length meshes
* fix: make it easier to transfer unsharded to sharded meshes
* fix(skeletons): better centering in meshes
4.10.0
------
* release(4.10.0): add truncate-scales flag to xfer and fixes z chunk size
* feat(image xfer): add truncate\_scales flag and parameter
* fix: handle larger z chunk size than initial
4.9.0
-----
* release(4.9.0): adds --encoding-level to image xfer
* feat: add encoding level to transfer tasks (#140)
* docs: update ChangeLog
4.8.0
-----
* release(4.8.0): -x works for SQS, don't encode degenerate meshes
* feat(cli): add message noting that sqs is waiting
* refactor(cli): use CloudPath param type instead of extra loc
* feat(cli): add CloudPath parameter class
* docs: note why the sqs empty function is complicated
* feat(sqs): make -x flag work for SQS (120 second wait before exit)
* fix: don't try to encode degenerate meshes
4.7.0
-----
* install: ensure trimesh has needed deps
* install: add mysql-connector-python to Dockerfile
* docs: fix spelling error
* refactor: change constant to appropriate variable
* perf: parallelize each shard access
* perf(skeletons): only fetch necessary skeletons from remote shard
* perf: don't download the whole remote shard for meshes
* install: update dependencies
* fix: wrong name for shard-computer dependency
* perf: much faster shard hash calculation
* feat(spatial\_index): add --allow-missing, parallel, max\_labels\_per\_shard
* build: issue with numpy 1.23.0 through 1.23.2
* build: build and push to dockerhub
* build: try pushing docker
* chore: update docker to python3.9
* Create docker-image.yml
* feat: add pyproject.toml
4.6.0
-----
* release(4.6.0): fixes several ccl issues and adds --dust argument
* ci: ensure numpy installed first
* docs: add dust\_threshold to documentation
* test: add py37 and py310 to test roster
* fix(ccl): alignment issue near boundaries
* test: check dust works
* feat(ccl): add dust threshold
* fix(ccl): thresholding operation was incorrect
4.5.5
-----
* release(4.5.5): fixes threshold and mip issues with ccl
* chore: update Changelog and AUTHORS
* fix: prior fix for mips was off by one
* fix(cli/ccl): don't crash if you don't specify a binarization threshold
* fix: relabel tasks should copy correct mip levels
4.5.4
-----
* release(4.5.4): fix ccl cli binarization thresholds
* fix(cli/ccl): convert thresholds to integers
4.5.3
-----
* release(4.5.3): lower memory CCL, fixed factor in xfer image tasks
* perf(ccl): reduce memory usage for computing relabeling
* fix: use factor arg within create\_transfer\_tasks (#138)
4.5.2
-----
* release(4.5.2): fixes typo in unsharded->sharded meshes
* fix(mesh/multires): misspelled variable
4.5.1
-----
* release(4.5.1): fixes fill missing for ccl
* fix(ccl): fill\_missing added to task creation routines
* chore: update version
4.5.0
-----
* chore: update ChangeLog
* feat: add fill\_missing to ccl
* docs: add CCL scripting section
4.4.0
-----
* release(4.4.0): CCL without sqlite requirement + binarization thresh
* perf: scale up CCL (#136)
* release(4.3.0): adds CCL capability
* feat: whole image connected components labeling (#135)
* docs: add CITATION.cff
4.2.1
-----
* release(4.2.1): fix mesh xfer, add BASIC support multires unsharded
* fix+feat: support multires meshes for mesh xfer
4.2.0
-----
* release(4.2.0): support for multi-resolution meshing
* fix: issues in unsharded multires
* fix: update when iteration finishes
* feat: add provenance info to unsharded
* feat+refactor: adds min\_chunk\_size to merge CLI interface
* feat: multiresolution meshes (#132)
4.1.3
-----
* release(4.1.3): fix DracoPy requirement
* feat: sort mesh fragments in a consistent order before combining
* fix: give sharded meshes tighter manifest bounds
* perf(meshing): use new ambidextrous zmesh to omit a conversion
4.1.2
-----
* release(4.1.2): fixes unsharded skeleton merging
* chore: update changelog
* Some inappropriate code at UnshardedSkeletonMergeTask (#130)
* fix: skeleton sharded merge not correctly passing spatial index arg
4.1.1
-----
* release(4.1.1): fixes dependency logic for DracoPy and zmesh
* chore: allow igneous to upgrade zmesh and dracopy for feature releases
4.1.0
-----
* release(4.1.0): adds min\_shards parameter
* feat: add min\_shards parameter to shard generation
* chore: update changelog
* fix(cli): normalize path for sharded skeleton merge
* refactor: correct type annotation
* feat: add --quiet option to igneous execute
* fix: add "none" to compress options
4.0.0
-----
* release(4.0.0): reorganizes CLI, fixes contrast tasks, adds to CLI
* fix(cli): missing argument and path normalization for contrast
* fix: contrast normalization task wouldn't handle non-chunk aligned tails
* docs: add contrast commands to cli
* docs: update image based commands
* feat(cli): add "igneous image contrast" menu
* BREAKING: redesign: reoganize the image CLI commands
* docs: update sharded skeleton example
* fix: draco compression level was set too low by default
* fix: set bounds for contrast normalization task
3.2.0
-----
* release(3.2.0): Adds spatial index operations for skeletons
* chore: update ChangeLog
* feat: add spatial-index command to skeletons
3.1.0
-----
* release(3.1.0): adds delete for meshes and skels, xfer for skels
* feat: add skeleton xfer (#125)
* feat: delete meshes (#124)
* refactor: remove unnecessary imports
* chore: update ChangeLog
* docs: update the MacOS reference
* docs: update obsolete reference to task\_creation.py
* docs: mention MapBuffer in related projects
3.0.0
-----
* release(3.0.0): update xfer and downsample commands to have --encoding
* chore: update ChangeLog
* redesign(cli): change --cseg and --compresso flags to --encoding (#123)
2.0.0
-----
* release(2.0.0): offsets meshes to align with neuroglancer, closes faces
* docs: discuss why voxel\_centered=False is set for graphene
* fix(meshes): closed faces on boundary and +0.5 voxels (#121)
* perf: streamlined manifest generation on filesystem (#119)
* docs: add comment to mesh xfer
1.0.0
-----
* release(1.0.0): Adds mesh xfer command
* feat: transfer unsharded meshes to sharded meshes (#116)
* fix: save mip level of meshing during transfer
* feat: adds igenous mesh xfer to CLI
* chore: credentials.tar.enc is no longer used
0.9.3
-----
* fixtest: accomodate additional generated mip levels
* release(0.9.3): fixed a more general version of vol < chunk size
* fix: handle vol size < chunk\_size more generally
0.9.2
-----
* release(0.9.2): fixes errors caused by specifying bounds
* docs: clarify which mip level to use for xrange,yrange,zrange
* fix: each reference to get\_bounds should have a bounds\_mip argument
* fix: unused variables without definition
0.9.1
-----
* release(0.9.1): can downsample when vol size < chunk size
* fix: generate downsamples where the volume size < chunk\_size
* wip: unsharded to sharded transfer
* feat: transfer unsharded meshes to sharded meshes
* docs: examples of igneous execute -x
* docs: acks, minor bugs, mention DracoPy as a related project
0.9.0
-----
* release(0.9.0): sharding upgrades, mesh spatial index gen, CLI updates
* feat: sharded meshes (single res using trivial multi-res) (#112)
* chore: add test-suite workflow (#115)
* chore: add trimesh, bump cloud-volume in requirements
* (fix): Add kwarg for updated get\_bounds (#113)
* refactor: moved compute\_bounds script to CLI
* feat(cli): adds "igneous design bounds" command
* refactor: remove dead logger code
* fix: missing declaration for reduce
* fix: missed change of function name in refactoring
* feat: bare-bones unsharded single-res multi-resolution meshes (#111)
* (feature): Add bounds\_mip to get\_bounds (#108)
* feat(xfer): add no\_src\_update flag
* fix: don't try to write provenance files to http
* docs: add description of broken test
* fixtest: use the right content encoding
* test: check that broken dataset sums to 20 bits
* chore: update changelog
0.8.2
-----
* release(0.8.2): fixes bug in computation of shard bits
* fix: rounding errors in computing shard bits (#107)
* chore: update CHANGELOG
0.8.1
-----
* release(0.8.1): fixes bug in sharded transfers and downsamples
* fix(transfer): sharding shape computation did not account for minishard bits
0.8.0
-----
* release(0.8.0): adds --clean-info flag
* fix: QuantizeTask was broken
* feat(xfer): add --clean-info which removes skeleton and mesh metadata
* docs: update how to use zrange
0.7.0
-----
* release(0.7.0): add xrange, yrange, and zrange to downsample
* chore: delete travis
* fix(cli): pass bounds the sharded downsampling
* feat: add bounds to sharded downsamples
* feat(cli): adds xrange, yrange, and zrange to downsample
0.6.0
-----
* release(0.6.0): adds igneous downsample --sharded
* chore: remove travis ci badge (travis no longer does free open source builds)
* docs: add igneous pypi badge
* feat: shard downsampling (#106)
* refactor: change TouchTask and QuantizeTask to use queuable
* feat: add sparse support to image downsampling
0.5.0
-----
* release(0.5.0): Adds --sharded to xfer task
* chore: remove unused environment variables
* docs: info on sharded scripting and delete CLI
* chore: delete obsolete dockerfile
* feat: create image shards (#105)
0.4.1
-----
* fix(cli): --cseg was not working
0.2.0
-----
* release(0.4.0): updates to xfer and adds design command
* feat(TransferTask): fix bugs, increase performance, CLI updates (#101)
* refactor(DeleteTask): change to use queuable functions
* chore: fix deployment of README
* refactor: move infrequently used tasks to obsolete.py
0.1.0
-----
* refactor: remove [deflate] extra (it's standard nowadays)
* feat: setting up pypi package
* fix: mesh stealing bug (#96)
* chore: dracopy version number in wrong position
* chore: bump cloud-volume to match floating resolution release
* feat: support floating point resolutions (#91)
* feat(cli): add "rm image" command for deleting images
* docs: add skeleton forge and merge to README top
* docs: add "comma delimited" to CLI error message
* Implement param type for cli tuple options (#94)
* docs: describe CLI better
* chore: whitespace
* chore: bump cli version to 0.3.0
* feat(cli): add sharded merging to the skeleton command set
* feat: add unsharded skeleton merging
* feat: add max\_cable\_length arg to unsharded skeletons
* feat(cli): add skeleton forging
* feat: add fill\_holes to skeletonization options
* docs: make it easier to understand how to use the CLI
* docs: make mesh comment clearer
* feat(cli:mesh): adds mesh commands
* docs: realized we never say what Igneous does in the intro
* docs: make sure people understand the CLI tool isn't all of igneous
* docs: show how to use the CLI
* docs: describe xfer tasks better
* feat: add CLI with downsample, transfer, and execute functions
* chore: updated license Trove identifiiers
* feat: add sparse downsamples to TransferTask
* chore(MeshTask): switch from pickle to mapbuffer
* refactor: remove unnecessary imports
* fix(DeleteTask): do not redirect delete tasks
* merge: new max\_cable\_length changes
* feat(create\_sharded\_skeleton\_merge\_tasks): adds max\_cable\_length
* fix: syntax error
* feat: add max\_cable\_length to task creation
* refactor: TransferTask and DownsampleTask are queueable functions
* fix: missing z\_bits from graphene manifest generation
* fix: factor was not preserved in create\_downsample\_tasks
* fix: finally update /secrets to /root/.cloudvolume/secrets
* fix: update deployment.yaml for K8s 1.16+
* fix(TransferTask): skip\_downsamples was not being respected for scales
* feat: add max\_cable\_length argument to shardedskeletonmerge
* perf: use mapbuffer validate b/c it's faster
* perf: use mmap for sharded skeleton merge when possible (#82)
* perf: why even spend the time checking label in frag
* perf: start using MapBuffer for sharded skeletons
* refactor: change BlackoutTask to use queueable functions
* fix: wrong variable name
* perf: another attempt at reducing memory usage
* perf: attempt to reduce memory usage for ShardedSkeletonMerge
* feat: add time argument to --loop (removes --no-loop)
* chore: make deflate an optional dependency
* fix: add total argument to create progress bar for .labels uploads
* fix: don't jsonify twice
* fix: ensure correct mip is used
* feat: add performance and progress to sharded skeleton generation
* perf: faster gzip decompression
* chore: upgrade to py38
* fix: Meshing examples require tq.execute() after each phase
* docs(README): update description of supported configurations
* refactor(tasks.py): use CloudFiles instead of Storage
* refactor(skeletonization): use CloudFiles instead of Storage
* refactor: replace Storage with CloudFiles in meshing tasks
* refactor(MeshManifestTask): switch to using CloudFiles
* feat(MeshManifestTask): adds support for mesh\_dir in task\_creation
* fix: remove unnecessary requirement oauth2client
* fix: stop uploading vertext\_types with skeletons
* fix(test\_downsample\_w\_missing): failing test
* refactor: no need to maintain two implementations of chunks
* fix: add deprecated -m flag
* feat: customize the number of parallel processes
* feat: igneous now tallys completed tasks for fq
* docs: fix name of create\_skeletonizing\_tasks
* chore: bump task-queue to latest compatible version
* Allow boto3 client parameters region\_name and endpoint\_url (#75)
* docs: show how to use FileQueue
* docs: show how to use "factor" in downsample tasks
* docs: show how to use the new TaskQueue
* test: add basic skeletonization task
* chore: bump task-queue to 2.0.0
* fix: bring task\_execution.py up to date with task-queue 2.0.0
* fix: errant self reference in create\_touch\_tasks
* feat: add support for fix\_avocados in Kimimaro
* fix: syntax error when shape or dtype is omitted from decode
* fix: use variable instead of 'gzip'
* feat: MeshTask supports brotli
* fix: add module prefix to create\_downsample\_scales
* fix: inputs to add\_scale
* fix: add num\_mips to provenance file
* refactor: remove unused build function
* fix+test: downsample tasks actually use the factor argument
* feat: add factor argument to task creation for downsample and transfer
* feat: add factor argument to Downsample and Transfer
* fix: propagate fill\_missing in GrapheneMeshTask
* fix: handle compressed segmentation correctly
* feat: ensure TransferTasks can be created with compress keyword
* docs: missing table entry
* feat: support brotli compression for downsamples and transfers (#66)
* fix: make it possible to visualize skeletons while performing postprocessing
* fix: problems with fusing skeletons due to integer truncation
* fix: remove caching from ShardedSkeletonMergeTask
* fix: remove pylru as we're not using persistent caches anymore
* perf: much more efficient ShardedSkeletonMergeTask
* docs: changed name of docker to remove :master
* feat: add precalculated $shardno.labels files to sharded task generation
* fix: don't crash when there's only one segid in the whole volume
* fix: wrote sign
* fix: ensure remapping code respects 'chunks\_start\_at\_voxel\_offset'
* fix: bug in stop\_layer calculation
* chore: don't dockerize data
* fix(ShardedSkeletonMergeTask): don't error on 0 skeletons
* perf: process empty GrapheneMeshTasks faster
* feat: L2 GrapheneMeshTask (#64)
* fix: typo in upload\_batch for MeshTask
* fix: remove extraneous printing from ShardedSkeletonMergeTask
* docs: new sharded and unsharded skeletonization
* feat: sharded skeleton synthesis as an Igneous task (#61)
* fix: wrong call to cloudvolume, fix remapping
* refactor(MeshTask): use fastremap for remap and move meshdir into function
* feat(TransferTask): allow creating flat segmentations from graphene (#63)
* perf(MeshTask): speed up dust removal
* feat: add timestamp parameter to MeshTask
* feat: mesh graphene volumes
* feat(ContrastNormalizationTask): Specify lower / upper fraction separately (#62)
* feat: add synapse tracing capability (#60)
* fix: add legacy mesh type to mesh info files
* fix: set cc3d safety factor to 1
* refactor(TransferTask): translation without unaligned writes (#58)
* Make updates for cloud-volume v1.0 & remove mesh info creation (#57)
* feat: add mip and chunk\_size to mesh info file
* feat: added more sophisticated checkpoint logic
* fix: increase number of default shard bits to 9
* feat: dataset specific checkpoint names
* feat: add checkpoint to skeleton merging code
* fix: send skeleton not label to complex\_merge
* perf: reduce memory usage for single process final merging
* perf: avoid storage overhead for unfused\_skeletons
* feat: use shared memory manager for parallel processing
* feat: special handling for parallel == 1 to allow line\_profiler
* perf: reduce memory usage during unpickling shard fragments
* perf: parallel sharded skeleton generation script
* fix: strip non-float32 vertex attributes from sharded skeletons
* feat: upload mesh info file with spatial\_index chunk\_size
* feat: add spatial\_index and sharded to create\_meshing\_tasks
* feat: add support for shard fragment mesh generation
* refactor: alphabetize meshing options
* feat: add support for generating a spatial\_index for meshes
* docs: describe mesh task
* fix: some issues with 3.7 invocations
* chore: update igneous to python37
* chore: update dockerignore to reduce image sizes
* Update README.md
* feat: actually upload the shard files
* wip: checking that something is output at all
* fix: update compute\_bounds.py to use current standards
* chore: cleanup .DS\_Store
* wip: script to generate sharded skeletons
* fix: ensure that the spatial index for skeletons is non-overlapping
* fix: issue correct spatial index chunk size
* fix: incorrect variable names
* feat: support for alternative write destinations for downsampling
* refactor: Combine DownsampleTask and TransferTask
* docs: updated documentation for Igneous
* fix: correct for resolution
* fix: update spatial index field for skeleton info file
* feat: upload condensed skeleton fragments (#54)
* feat: add delete\_black\_uploads and bgcolor to downsampling tasks
* chore: CloudVolume 0.59.1 has background\_color
* chore: upgrade tqdm in requirements
* feat: TransferTasks support different background colors
* fix: "fill\_missing" was not in proveneance data for downsampling
* docs: add parameters for downsample, transfer, delete
* feat: add support for delete\_black\_uploads to TransferTask
* feat: add fill\_missing option to SkeletonTask
* perf: improve speed of object masking in mesh task
* fix: compatibility with cloudvolume 0.51.0
* refactor: use postprocessing from kimimaro 0.5.0
* fix: stable\_prefixes -> prefix\_length
* feat: add object masking to skeleton tasks
* feat: enable shared memory on kubernetes depoyments
* feat: skeleton graphene hybrid tasks
* chore: bump kimimaro version
* fix: shape and offset passed by value in FinelyDividedTaskIterator
* feat: encode created meshes in draco format (#46)
* feat: parallel argument for skeletonization
* fix: parallel should be spelled self.parallel
* feat: add parallel, progress, and dust\_threshold to skeleton options
* feat: add nano to igneous containers for editing capability
* feat: add fill\_missing to create\_meshing\_tasks
* feat: add fill\_missing option to MeshTask
* refactor: use zmesh 0.2.0 idioms
* docs: add zmesh to list of related projects
* perf: 32-bit Triangle Representations for Meshes (#49)
* Update README.md
* docs: added "Related Projects" section
* refactor: remove incorrect comment in setup.py
* fix: make will\_postprocess json serializable
* chore: specify minimum Kimimaro version
* fix: remove debug code and restore provenance for SkeletonTask creation
* feat: support Kimimaro 0.3.1's fix\_border mode (#52)
* refactor: remove skeletontricks (kimimaro already has it) (#50)
* fix: create\_delete\_tasks wasn't handling boundaries correctly
* fix: remove progress bar from production skeletonization
* perf(MeshTask): early abort if all zeros
* perf: reduce average memory pressure from meshing (#48)
* docs: update skeletonization to remove incorrect info
* fix: add chunk alignment to QuantizeTask creation
* fix: TransferTask should use return rather than yield
* feat: support for parallel upload for grid-regular tasks (#47)
* fix: missing arguments to MeshTaskIterator and new fn parameter
* fix: dust threshold was in the wrong direction..
* fix: restore mistakenly removed import of 'os'
* feat: Skeletonization (#12)
* feat: add dust\_threshold option to MeshTask
* fix: report total length as int for MeshTaskIterator and DeleteTaskIterator
* feat: skip\_downsamples flag for TransferTask
* fix: set default meshing shape to (256, 256, 256)
* fix: remove noisy progress bar from MeshManifestTasks
* fix: remove redundant progress bar from MeshTask creation
* fix: add encoding to provenance file for DownsampleTasks
* fix: use prefixes correctly in hybrid mesh manifest task
* wip: hybrid graphene mesh manifests
* chore: help protect against accidental secret uploads
* chore: ignore test2.py
* revert+fix: clamp to volume bounds in TransferTask
* fix: cap volume bounds at correct mip level for TransferTasks
* feat: introduce tinybrain (#44)
* docs: update README to reflect new TaskQueue upgrades
* fix: TransferTask boundary clamping should be handled in task creation
* fix: dvol\_bounds needed seperate definition
* fix: create\_transfer\_tasks respect destination bounds rather than source
* feat: add encoding parameter to create\_transfer\_tasks
* perf: use node DNS over cluster DNS
* fix: missing reference to LOOP
* feat: create TouchTask and BlackoutTask
* chore: bump requirements for cloudvolume and taskqueue
* refactor: remove dead code in task\_execution
* refactor: modernize task polling
* chore: make sure "projects" directory isn't included in docker images
* fix: single slice contrast normalization tasks
* fix: QueueEmpty error
* fix: integer length for DownsampleTaskIterator
* refactor: remove dead fastremap code
* chore: remove PrintTask
* refactor: Convert task\_creation.py to Iterators and TaskQueue 0.13.0 (#42)
* fix: several errors introduced by spacing and renaming
* refactor: convert all remaining tasks to iterators
* chore: remove autodeploy, it's broken
* chore: add projects to gitignore
* fix: forgot to remove some legacy attributes on ContrastNormalizationTask
* fix: missed part of refactor
* refactor: additional conversion to iterators
* wip: add DownsampleTask, DeleteTask, and MeshTask to iterator paradigm
* wip: convert task\_creation to produce iterators
* docs: update downsample documentation
* fix: get\_bounds should have only an option for expand\_to\_chunk\_size
* feat: support bounds in provenance for contrastnormalizationtask
* chore: meaningless change to force deployment
* fix: adjust downsample shape when chunk\_size is specified
* feat: add bounds to ContrastNormalizationTask
* fix: ContrastNormalizationTask respects mip level for upload
* refactor(Dockerfile): Reduce Docker image size (#37)
* fix: queue\_pickle.py can save again
* Create LICENSE
* chore: pin python-jsonschema-objects
* feat: added minval, maxval, and level\_paths to ContrastNormalizationTask
* feat: added bounds parameter to luminance levels task
* chore: added resource limits to deployment memory
* feat: add levels\_path to LuminanceLevelsTask
* chore: use --password-stdin in .travis.yml (#33)
* fix: ensure python2 installs futures
* fix: python2 compatibility
* feat: can use different encoding on downsampled layers
* fix: don't commit provenance files to BOSS
* feat: TransferTasks support transfer from any mip level
* Update README.md
* fix: write bounds to provenance file w/o crashing for TransferTask
* feat: support mesh\_dir and cdn\_cache in create\_meshing\_tasks
* feat: use $USER intead of git user.email if not available
* feat(MeshTask): configurable mesh\_dir
* changed version.py to igneous\_version.py
* feat(MeshTask): cache\_control and parallel kwargs support
* fix: broken DeleteTask
* feat: make DeleteTasks more like DownsampleTasks
* fix: abort deletion if bbox volume is zero
* feat: add provenance support for DeleteTasks
* fix: upgrade PyOpenSSL
* feat: allow deleting only some mips with DeleteTask
* fix: add bounds to transfer task provenance
* feat: chunk\_size support in create\_downsampling\_tasks
* Remove layers added by RUN (#23)
* chore: bump cloudvolume to 0.33.4
* chore: remove intern from requirements, rely on cloud-volume version
* chore: bump intern to 0.9.9
* test: honesty in testing, only use one python version
* chore: bump python to 3.6
* chore: bump python version to 3.5 from 3.4
* feat: use git user email for provenance file
* fix: reduce default mesh shape to a reasonable size
* fix: change default queue type to SQS
* Add chunkflow InferenceTask and MaskAffinitymapTask (#17)
* feat: add bounds argument to create\_transfer\_tasks
* chore: remove inaccurate intern dependency
* feat: progress bars for task upload
* chore: bump cloud-volume version
* chore: fix some style issues
* fix(MeshTask): Set default low\_padding to 0 to allow for continuous mesh boundaries
* chore: bump cloudvolume to be compatible with matrix deployment
* fix(cMesher): revert change to max\_simplification\_error from #9
* chore: update requirements.txt, CloudVolume w/ matrix support
* fix: bbox in mip 0 coordiantes
* fix: autoresizing of bounding box needed to use shape not underlying
* feat: add bounds argument, documentation, and mip=0 to create\_downsampling\_tasks
* fix(MeshTask): \_update\_vertices() accounts for 'low\_padding' offset
* fix(setup): Add numpy includes
* feat: update QuantizeAffinitiesTask to QuantizeTask
* chore: bump cloud-volume to 0.28.2
* fix: add google-cloud-logging to requirements
* chore: bump cloud-volume to 0.28.1
* feat: Cython array remap code (#14)
* fix: shape computation error in create\_deletion\_tasks
* fix: compute mip coordinates correctly in DeleteTask
* docs: documentation for sparse flag in create\_downsampling\_tasks
* refactor: move pipeline running commands to pipeline.py
* feat: add sparse downsampling capability for segmentation (#15)
* Preparations for agglomeration meshing (#9)
* refactor: simplified mesh interface, now only need to call mesher.mesh(numpy\_array) (#13)
* fix: update queue\_pickle to python3
* fix: always set chunk\_size for transfer task even if info exists
* Update README.md
* Update README.md
* Update README.md
* fix: added provenance processing for MeshTask
* fix: updated deployment.yml to reflect SQS usage
* docs: some rewording
* docs: added info on running igneous in the cloud
* docs: local multicore processing without the cloud
* chore: bump task-queue to make LocalTaskQueue available
* fix destvol dtype bug for hypersquare consensus task
* feat: automatic info file with adjustable chunk size for TransferTask
* chore: bumped python task queue to version compatible with multi-secrets
* chore: bugfix cloud-volume version
* fix: handle boundary shape correctly for ContrastNormalizationTask
* fix: handle black slices correctly in ContrastNormalizationTask
* docs: Documented downsampling, deletion, transfer, meshing, and contrast correction
* fix: ensure Python 2 can perform accurate division in ContrastNormalizationTask
* feat: ContrastNormalizationTask (#7)
* chore: upgrade cloud-volume, tqdm, and numpy
* fix: write to mip level dir for LuminanceLevelsTask
* feat: created LuminanceLevelsTask (#5)
* chore: bump cloud-volume to 0.19.1
* fix: ensure delete tasks stay within volume bounds
* feat: move provenance email to variable
* chore: bump cloudvolume to 0.15.2
* fix: added missing properties to transfer task provenance
* Improve docs a bit (#4)
* chore: bumped cloudvolume version
* chore: clean up
* fix: provenance uploads correctly for transfer tasks
* fix: cast lease\_seconds to int
* fix: added LEASE\_SECONDS to secrets.py
* feat: added ability to specify lease seconds
* chore: bumped cloudvolume to 0.10.3
* fix: fixed DeleteTask for higher level mips
* fix: HyperSquareConsensusTask working for e2198
* feat: HyperSquareConsensusTasks are now 64bit
* fix: correctly reference SQS\_URL
* chore: upgrade python-task-queue
* feat: add support for SQS
* feat: added jittered exp backoff for task\_execution
* chore: bumped cloud-volume version
* feat: added support for countless3d (#2)
* feat: marching cubes now handled uint64 (#3)
* feat: added cascade function to task\_creation
* perf: improved memory efficiency and performance of countless downsample
* feat: sanely compute the downsample chunk size when preserve\_chunk\_size=False
* chore: remove \*.pyc files from test as well
* fix: reference seunglab/igneous:master b/c that's what travis deploys
* feat: default queue is google pull-queue rather than appengine
* chore: bumped click version in requirements.txt
* chore: delete \*.pyc files
* fix: improve context swtiching on processor
* chore: bump python to 3.4 for better performance
* feat: queue\_pickly.py script can save or reload task queues
* feat: added multiprocessing to task execution
* fix: python3 compatibility for creating mesh manifest tasks
* feat: able to use appengine queue
* feat: added example deployment file for kubernetes
* chore: update python-task-queue to 0.1.7 for appengine support
* fix: push to seunglab/igneous
* fix: install in editable mode to expose \_mesher.so
* fix: include \_mesher.cpp in travis builds
* feat: added DeleteTask and PrintTask
* test: adding debug statement to see what's going on in travis
* test: updating travis file with encrypted docker information
* fix: made scripts python3 compatible
* chore: configured dependencies to enable docker build
* perf: sped up tests
* fix: python 2/3 compatibility for version.py
* fix: several installation issues + multiple travis python versions
* fix: use relative paths instead of absolute in setup.py
* fix: passing tests in python3
* docs: added Travis CI indicator to README.md