We are currently working on porting this changelog to the specifications in Keep a Changelog. This project (loosely) adheres to Semantic Versioning.
- Added
requestkw
toub.download
.
- Minor test issues.
ub.IndexableWalker.diff
for empty inputs- Bug in
memoize_method
which could produce incorrect results if methods from different instances are assigned to variables. - Rare error in
test_copy_directory_cases
test due to list sorting.
- Added module name printout to
schedule_deprecation
ub.hash_data
now supportsdatatime.datetime
,datetime.date
,decimal.Decimal
objects.
- Add
ub.IndexableWalker.diff
- Added workarounds for copy / symlinks via
ub.Path
andub.symlink
on pypy. ub.import_module_from_path
now correctly acceptsPathLike
objects.ub.modname_to_modpath
fixed in cases where editable installs use type annotations in their MAPPING definition.
- Support for UNIX special permission (suid/sgid/svtx) codes in
Path.chmod
.
- Moved windows dependencies from requires to optional. Windows users that make use of these will need to update their ubelt install or explicitly depend on them as well.
- New wrapper around
pathlib.Path.chmod
inubelt.Path.chmod
. Can now specify string codes like "u+x" or "+rw". Old stat logic works as it previously did.
- Allow the argument to
ubelt.cmd
to be aPathLike
object, which we will expect to be an executable.
ub.modname_to_modpath
now handles cases where editable packages have modules where the name is different than the package.- Fixed deprecated usage of
ast.Num
- Add backend option to
highlight_code
which can be "pygments" or "rich". - Support for Python 3.12
- Improve speed of inplace dictionary set operations.
- Align in the case of
nobraces=1
forubelt.urepr
.
- Bug where the first call to
ub.hash_data
would error if it was given a ndarray with object type. - Actually exposed
ChDir
.
- Docs and typing improvements
- Skipped a release version due to a bad github tag.
- Removed experimental warning from
Path.copy
andPath.move
; they are now well enough tested. - The
util_format
submodule has been renamed toutil_repr
. Theutil_format
is now a deprecated alias forutil_repr
. - Stub out merge methods for SetDict.
- Renamed
FormatterExtensions
toReprExtensions
, with the former now being a deprecated alias. ubelt.cmd
now returns an object that ducktypessubprocess.CompletedProcess
.ubelt.cmd
now contains acapture
argument which will disable capturing of output.ubelt.cmd
...detaching now only prints if verbosity is > 1
- Exposed
ChDir
. - Removed usage of deprecated
pipes
module ubelt.cmd
return dictionaries are now more consistent
- The
recreate
argument toensuredir
- Added
ChDir
toutil_path
. - Add transient option to
JobPool
so references to futures are released after they are yielded to the user. - Added "base32" as an option for
hash_data
. Note that it is unpadded, hence non-RFC compliant. - Added
pattern
as a convenience option toubelt.Path.ls
.
- Updated vendored ProgIter to match progiter 1.3.0
- The download progress bar now gives more relevant information and updates less frequently.
IndexableWalker.allclose
can now take unwrapped objects.
ubelt.cmd
now respectscwd
whensystem=True
.
- Support for FreeBSD in
util_platform
ub.Path.copy
andub.Path.move
- Tentative Python 3.11 support
ub.urepr
which isub.repr2
with new defaults.ub.IndexableWalker.allclose
method to replaceindexable_allclose
ub.schedule_deprecation
can now accept the strings "soon" or "now" for "when" arguments.ub.schedule_deprecation
can now acceptstacklevel
as as an argument.ub.Path.apppdir
can now be called without arguments.- Deprecate
TempDir
ub.Timer
can now acceptns
as a keyword to enable nanosecond resolution.ProgIter.format_message
return value has changed.ub.Path.mkdir
now returns itself- Speedup
dict_hist
in the basic case by 2x
- Issue in
indexable_allclose
where tolerances were not respected. - Issue in
modname_to_modpath
with exclude and editable installs. - Incompatibility with pathlib in
PythonPathContext
- Fixed issue in progiter with line clears
- Issue in
import_module_from_modname
with editable installs where it would try to match modules that had the requested module as a prefix. - The
timeout
argument is now respected in the thread / process case ofJobPool
ub.indexable_allclose
. Useub.IndexableWalker.allclose
instead.
- Add
ubelt.util_dict.UDict.take
. - Added
__add__
,__radd__
,startswith
, andendswith
methods toubelt.Path
to make it a better drop-in replacement for string based paths.
- Reverse dunder methods of
ubelt.SetDict
now promote the type. - Add
cls
keyword argument toubelt.SetDict
methods. - Deprecate:
ensure_app_cache_dir
,ensure_app_config_dir
,ensure_app_data_dir
,get_app_cache_dir
,get_app_config_dir
,get_app_data_dir
,readfrom
,writeto
,ensure_unicode
. - Initial work on new Path methods for move and copy.
- CacheStamp.renew now does nothing and returns None if the stamp is disabled.
- AutoDict, SetDict, and UDict are now always ordered. In Python 3.6 it inherits from OrderedDict. Otherwise it uses regular dict, which will be ordered in 3.7+
- AutoDict now inherits from UDict.
- PathLike objects can now be used in args to ub.cmd when the command is an iterable.
- deprecate AutoOrderedDict, which is now indistinguishable from AutoDict
- Tentative fixes for new
__editable__
based install path withub.modname_to_modpath
- Implemented inplace and reverse versions of dictionary set operations
- Added copy to setdict
- Added keywords argument to
ub.compatible
. - Added
warncls
argument toub.schedule_deprecation
. - Experimental SetDict, UDict and aliases sdict, udict.
- Race condition on win32 in
ubelt.symlink
- Issue with
ubelt.memoize_method
where the method name and docstring were not wrapped correctly. - The
timeout
argument now works correctly withub.cmd
whentee=True
. - Added
appname
toubelt.download
which was supposed to exist as indicated by the docs, but didn't. - The resources used by
ubelt.cmd
are now properly closed.
ub.compatible
no longer errors on positional only functions, instead it returns the keyword-compatible arguments.- An issue in
ubelt.symlink
with unintuitive behavior when an empty string was given as the link path. This now raises an error. - The main implementations of
ubelt.sorted_vals
andubelt.map_vals
were renamed toubelt.sorted_values
andubelt.map_values
, but the old names are still available as aliases. - Positional arguments in
Path.augment
have been modified. - In
Path.augment
, deprecate overloadedsuffix
and introducestemsuffix
as an alternative. - Added cls to a lot of util_dict funcs
- Added new module
util_deprecate
with the functionschedule_deprecation
, which is generally useful for library maintenance.
- Fixed issue where ubelt Cacher triggered its own warnings
- Fixed deprecated usage of LooseVersion
- Tentative deprecation or backwards incompatible change in ub.Path.augment with suffix or prefix keywords
- Removed warning from
ubelt.Cacher
when depends is not specified. ub.timestamp
/ub.timeparse
now respects adefault_timezone
argument and handlesdatetime.date
objects.- Type stubs are now included in the distribution
- Issue #113, where a
ub.find_exe
test failed on Gentoo. Fixed by #114 - Issue where older versions of CacheStamp would be interpreted as 1.1 stamps.
- New method:
ub.timeparse
can parse the result ofub.timestamp
into adatetime
object. Can optionally usedateutil.parser.parse
under the hood. ub.Path.ls
a convenience function that aliaseslist(path.iterdir())
.ub.Path.walk
to wrapos.walk
.
- Register
pathlib.Path
withub.repr2
- Can now register global
ub.repr2
extensions viaub.repr2.register
- Can now register global
ub.hash_data
extensions viaub.hash_data.register
- Removed deprecated arguments from
ubelt.cmd
. ub.CacheStamp
will now check the mtime and size to quickly check if the products have changed and force expiration.ub.CacheStamp
now takes anexpires
keyword arg, which will keep the cache valid only for the specified amount of time.ub.CacheStamp
now takes anhash_prefix
keyword arg, which will check that it matches the hash of the product.ub.cmd
now has asystem
argument for modularity withos.system
.ub.cmd
now accepts atimeout
argument (tee support is pending).ub.JobPool
now contains a protected_prog
variable allowing the user finer-grained progress controls.ub.JobPool
now contains a convenience methodjoin
that executes all jobs and returns a list of results.ub.timestamp
can now accept adatetime
object as an argument, and will return the timestamp for that object.- The
ubelt.util_download.grabdata
function now usesCacheStamp
instead of implementing its own stamp solution. - The
ubelt.util_hash.HashableExtensions
implementation was updated to usefunctools.singledispatch
instead of the custom solution. This seems faster and should not have any API impact.
product
andcfgstr
arguments toCacheStamp.expired
product
andcfgstr
arguments toCacheStamp.renew
- Passing
hasher
as an instance to functions likegrabdata
orCacheStamp
can cause unexpected hashes as they may be used more than once.
ub.hash_data
now recognizes subclasses of registered types.ub.timestamp()
has been outputting incorrect (negated) UTC offsets. This is now fixed.ub.timestamp()
now works correctly when the year has less than 4 digits.
- Bug where six was used but not listed as a dependency. Six is now removed as a dependency.
- Fixed out of date docs in some places.
- :func:
ubelt.Path.appdir
which functions like theget_app_*_dir
methods inutil_platform
. - Add
tail
argument to :func:ubelt.Path.augment
and :func:ubelt.util_path.augpath
- Add json
backend
option to Cacher.
IndexableWalker
behavior has been changed, each timeiter
is called it resets its global state.- Remove support for Python 2.7 and Python 3.5
- Removed deprecated functions scheduled for removal.
- :func:
ubelt.util_dict.dict_diff
now preserves original dictionary order in Python 3.7+. ub.hash_data
can now hash slice objects.- INTENTION OF BREAKING CHANGE NOTIFICATION:
ubelt.util_format.repr2
may no longer sort dictionaries by default. Looking into a backwards compatible way to work around this.
- More
ubelt.Path
extensions fordelete
- Add
timeout
parameter toubelt.download
- Modified default
ubelt.Path
behavior fortouch
to return a self-reference for chaining
- Added
ubelt.Path
, as an extension and quicker-to-type version of pathlib.Path with extra functionality. - Added
progkw
as argument toJobPool.as_completed
to control progress reporting - Added
progkw
as argument toub.download
/ub.grabdat
to control progress reporting - Added
util_zip
with thezopen
function. Access a file inside a zipfile with a standardopen
like interface.
ubelt.hash_data
now handles non-numeric float values.ubelt.chunks
now works correctly when nchunks is specified.
- Changed default of
_hashable_sequence
types
arg from True to False to be consistent, but kept existing types=True behavior in hashable extensions. Changes should be backwards compatible, but in the future we may introduce a breaking change to make hash behavior more consistent.
- Added pyi type annotation files. (Used a custom script to export docstring type annotations)
- Added
default
keyword argument to signature ofub.peek
- Added
map
function to the executor classes. ub.Executor
now correctly returns itself from__enter__
- Docstrings now have better type annotations
- ProgIter had a bug in
time_thresh
, where it was never used (modified adjustment rules). - Fixed performance regression in ProgIter
- New CI GPG Keys: Erotemic-CI: 70858F4D01314BF21427676F3D568E6559A34380 for reference the old signing key was 98007794ED130347559354B1109AC852D297D757.
- Verbose test from symlink previously showed "real -> link" which makes no sense because a link should be the object that "is pointing". Thus it now shows "link -> real"
ub.download
should now generate less stdout text- New in-repo "dev" benchmarks
- Documentation fixes
- new hashing 36-character base with alias (alphanum / abc123 / 36)
- Added "compact" argument to
ub.repr2
- added candidate utilities:
named_product
,varied_values
toutil_dict
- added candidate utilities:
compatible
toutil_func
- Added
util_indexable
andIndexableWalker
(ported from kwcoco) - Added
util_futures
withub.JobPool
andub.Executor
(ported from kwcoco) - Added
util_download_manager
with simple implementation ofub.DownloadManager
- Added candidate functions to
ubelt
proper
ubelt.download
now errors earlier if the parent directory does not exist- PyTest no longer throws warnings
- Fixed issue with
download
and ByteIO objects in 3.8 - Bug in Python 3.8+ on win32 that did not account for the change on
os.readlink
behavior
-
Modified corner cases in
ub.repr2
to move towards behavior that is easier to reason about. -
Remove support for Python 3.4
blake3
is now an optional hasher
-
ubelt.hash_data
can now hash dictionaries and sets by default. -
increased test speed
-
Internal change in how external hashers are maintained.
- On windows colorama init is no longer called if it was ever initialized before. This fixes rare infinite recursion bugs when using pytest.
- Added
maxbytes
parameter tohash_file
to allow for only hashing a prefix.
- Docs seem to be building correctly now
-
Made import time 13x faster (was 109680, is now 8120) by using lazy external type registration in
util_hash
and removing other eager imports. -
Removed import dependency on six. There is still a runtime dependency, but we are moving away from six. This is a first step to deprecating Python2 support
-
Changed default of "blocksize" in
hash_file
to2 ** 20
based on benchmarks. -
Removing Travis-CI, will soon migrate to Circle-CI
- Added
meta
anddepends
toCacheStamp
to agree withCacher
ProgIter.step
can now accept theforce
keyword argument to force displayProgIter.step
returns True if the display was written
- Bug in
dict_isect
where order was not taken into account - Bug in
ProgIter
display frequency adjustment
- Tweaked display frequency adjustment in
ProgIter
ProgIter
no longer displays wall time by default. Setshow_wall=True
to regain this functionality. When true this now shows the date and time.
ub.repr2
now accept type name strings at register time (which makes it easier to lazy-load heavy libraries)ub.repr2
now handles pandas.DataFrame objects by defaultub.repr2
now accepts thealign
keyword arg, which will align dictionary kv separators.- functions in
ub.util_color
now respects a globalNO_COLOR
flag which prevents ANSI coloration.
ProgIter.step
now respects update freq, and will not update the estimates if too few iterations have passed. This preventsub.download
from generating extremely large amounts of standard out.ub.Cacher
now reports the file size of the cache file.ub.Cacher
now defaults to the latest pickle protocol (-1), which may cause compatibility issues.
ProgIter
now correctly checks if it needs to displays a message on every iteration.- Fixed uninitialized
_cursor_at_newline
variable inProgIter
.
ub.repr2
now encodes inf and nan asfloat('inf')
andfloat('nan')
to allow output to be evaluated.ub.grab_data
now uses the hasher name in the cached hash stamp file.
- Fixed issue in setup.py that broke the previous release.
NOTE: THIS RELEASE WAS BROKEN DUE TO AN ISSUE WITH THE SETUP SCRIPT
dpath
andfname
keywords to theub.download
function.modname_to_modpath
can now find modules referenced by egg-link files.ub.sorted_keys
andub.sorted_vals
for sorting dictionaries
ub.download
now acceptssha256
andmd5
hashes.
- The argument names in
ub.group_items
,groupids
was changed tokey
. - The argument names in
ub.dict_hist
.item_list
was changed toitems
,weight_list
was changed toweights
. - The argument names in
ub.flatten
.nested_list
was changed tonested
- Added
check
kwarg toub.cmd
, which when True will raise aCalledProcessError
if the exit-code is non-zero. - Added support for pypy.
- Moved
timerit
to its own module.
- Fixed corner case where
util_hash
raised an import error when python was not compiled with OpenSSL.
- Removed the
NoParam.__call__
method. This method should not have been defined, and by existing it caused issues when usingNoParam
as a column-key in pandas.
- Timerit now has 3 new properties
measures
,rankings
, andconsistency
. These keep track of and analyze differences in timings between labeled timerit runs. ub.take
now acceptsdefault=NoParam
keyword argument.
- Substantially improved documentation.
- The following functions are now officially deprecated:
dict_take
- The following functions are now officially deprecated:
startfile
,truepath
,compressuser
,editfile
,platform_resource_dir
,get_app_resource_dir
, andensure_app_resource_dir
,dict_take
- Improve docs
Timerit
andProgIter
are back, remove dependency on the external modules.
PythonPathContext
now works in more corner cases, although some rarer corner cases will now break. This trade-off should be a net positive.
- Added
dpath
as an argument toub.augpath
- Custom extensions for
ub.hash_data
are fixed. Previously they were not passed down more than a single level. - The
convert
option forub.hash_data
was previously not hooked up. - Correctly expose
dict_diff
- Fixed issue in
ub.augpath
wheremultidot
did not preserve the original extension
ub.Cacher
no longer ensures that thedpath
exists on construction. This check is delayed untilsave
is called.ub.CacheStamp
now accepts theenabled
keyword.modpath_to_modname
now properly handles compiled modules with ABI tags.
- Add
ub.dict_diff
, which removes keys from a dictionary similar toset
difference. - Add
ub.paragraph
, which helps with writing log messages - Add some benchmarks
- Add lots more documentation.
ub.identity
now accepts*args
and**kwargs
and defaults the first argument toNone
, but still only returns the first argument.- The
sort
kwarg ofub.repr2
can now accept a callable, which will act as a key to thesorted
function ub.hash_data
now accepts theextensions
kwarg, which allows the user to define how particular types are hashed.
- Fix GH #53
- the
index
argument inimport_module_from_path
is now correctly used.
- Fixed bug in
ub.dict_hist
whenordered=True
. (half of the keys would be lost). Also effecteddict_take
. platform_data_dir
now correctly raises an exception when the operating system is unknown.
- Add
memoize_property
ub.cmd
now reportscwd
on exception- Reworked requirements to minimize dependencies.
- The
xxhash
andpygments
dependencies are now optional. - The testing dependencies are now optional.
- new tests
- add
util_stream
- Fixed issue in
ub.download
with bad content header urls
ub.platform_cache_dir
andub.platform_config_dir
now respect XDG environs on Linux systems.
ub.download
can now acceptfpath
as either a file path or aio.BytesIO
objectub.FormatterExtensions.register
can now accept a type or tuple of types.
ub.platform_resource_dir
is deprecated in favor ofub.platform_config_dir
.
ub.repr2
now accepts negative values fornewlines
, which means use newlines until the current height is only-newline
.ub.repr2
now keeps track of nesting depth from the bottom- Make result
ub.memoize_method
appear more like a bound method.
- Add custom extensions to
ub.repr2
and exposeub.FormatterExtensions
- Add
dict_isect
toutil_dict
.
- Fixed misspelling in docs
- Fixed misspelled detach kwarg in
ub.cmd
(the olddetatch
argument is now deprecated and will be removed)
- Add class variable
FORCE_DISABLE
toub.Cacher
- Add the
xxhash
algorithm as an option toub.hash_data
- Add
ub.peek
- 4-letter syntactic sugar forlambda x: next(iter(x))
- Add
key
toub.find_duplicates
- Renamed first argument of
ub.chunks
from sequence to items - Improved type hints in google-style docstrings
ub.cmd
verbose >= 3 now uses nicer Unicode characters if possible
- Fixed GH#41
- Add verbose flag to
ub.CacheStamp
ub.group_items
argument names have changed, and it can now take a callable as the second argument. Thesorted_
argument is now deprecated.- Symlink now reports location of old target when the new target does not match
- Docstrings now uses
PathLike
as the type for arguments and attributes that should be considered paths (note strings are still accepted). ub.download
will now keep a potentially corrupted file if the hash does not match.ub.grabdata
will compute the hash of an existing file on disk if the .hash stamp is missing to try and avoid re-downloading the file.- Improved efficiency of
ub.argmax
- added
ub.expandpath
- Certain imports are now lazy to optimize startup time
- change
ub.cmd
tee
parameter totee_backend
(BREAKING CHANGE) - change
ub.cmd
verbout
parameter totee
(BREAKING CHANGE) import_module_from_path
can now handle zip-imports where the zip-file is followed by a slash rather than a colon
tee
parameter fromub.cmd
totee_backend
verbout
fromub.cmd
ub.find_exe
- a python implementation of whichub.find_path
- finds matching files in your PATH variablesub.CacheStamp
- Replace in-house implementation of
OrderedSet
with the ordered-set PyPI package. ub.download
now acceptshash_prefix
andhasher
args.ub.hash_file
now acceptstypes
argsub.augpath
now acceptsmultidot
argsub.cmd
now acceptscwd
andenv
args- Changing default behavior of
util_hash
. (BREAKING CHANGE)- Default of
ub.Cacher
maxlen
changed to 40 for sha1 considerations - Default of
ub.hash_data
base
changed fromabc
tohex
- Default of
ub.hash_data
types
changed from True to False. - Moved argument position of
hashlen
to the end.
- Default of
- Remove
ub.OrderedSet
.extend
ub.NoParam
is now Falsey
ub.import_module_from_path
can now import modules within zip-files
ub.PY2
andub.PY3
. Usesix
instead.
ub.dzip
now accepts a backend dict class as a keyword argumentOrderedSet.intersection
can now handle a single argumentTimerit
num
now defaults to 1- Add function
print
to Timerit
- Fix timezone issue with negative time-zones
- Move internal
__init__
auto-generation logic to newmkinit
repo - Network tests no longer run by default and require
--network
- Add
ub.argmin
andub.argmax
ub.Cacher
can now be used as a decorator.- Rename
util_decor.py
toutil_memoize.py
- Add
key
argument toub.unique
andub.unique_flags
- Add
ub.argunique
import_module_from_path
now prefers the path module when there are name conflicts- Fix
ub.repr2
precision with numpy scalars - Add
ub.dzip
- Add
inject_method
toutil_func.py
. - Add
allsame
- simplified dynamic imports
memoize_method
now handles kwargs- Can now update
ProgIter
description on the fly - Add methods to
OrderedSet
to complete the set API (e.g.intersection
,difference
, etc...) - Can now index into an
OrderedSet
using a slice - Add
appname
keyword argument tograbdata
- Add
extend
to ordered set API - Increase
tqdm
compatibility withProgIter
- Fixed issue with
OrderedSet.union
where it ignoredself
- Fixed issue with
OrderedSet.union
where__eq__
andisdisjoint
were wrong - Fix issue with
ub.repr2
dictionaries with newlines in keys - Fix issue with relative paths and symlink
ub.iter_window
- Spelling: changed the
Timer.ellapsed
attribute toTimer.elapsed
. - Verbosity of
Timer
andTimerit
now depends on if a label was specified. Timer.tic
now returns a reference to theTimer
instance.
- Remove
util_stress
, it was out of scope.
hash_data
can now acceptOrderedDict
inputdict_union
now returnsOrderedDict
if the first argument is one
- bug in
hash_data
where negative integers did not work.
OrderedSet
/oset
- Add
symlink
function that works on UNIX and Windows*. (*if use has symlink permissions, it works just like UNIX without caveats. Otherwiseub.symlink
falls back to using junctions and hard links, which should still work mostly the same, exceptos.path.islink
andos.readlink
will not work among other minor issues).
- Add base to
augpath
ub.delete
now treats nested junctions as symlinks, unlikeshutil.rmtree
.
- Add
numpy
support toub.repr2
- Changed
ub.Timerit
.call API to return a reference to the Timerit object instead of of the average seconds. Note this change is backwards incompatible.
ub.hash_data
andub.hash_file
for easy hashing of arbitrary structured data and file.ub.dict_union
combines multiple dictionaries and returns the result.
ub.Timerit
reports better measures of expected time.- New argument
total
toub.chunks
lets you specify how long an iterable is iflen
is not available (for generators)
- Add
ub.TempDir
- Add
ub.import_module_from_path
- Add
ub.import_module_from_name
- can now choose
ub.cmd
tee backend (select or thread) on POSIX. ProgIter
now supports a moretqdm
-like API- Add standard deviation to
timerit
- Minor enhancements to
ub.Cacher
- fixed unused argument
chunksize
inutil_download
ub.cmd
tests now work on windows- terminal colors now work on windows
- Remove most of the
static_analysis
module. Use code in xdoctest for now. Note: some of this functionality may return as general utilities in the future, but the existing constructs were only needed for doctests, which are now done via xdoctest.
ub.truepath
ub.iterable
util_func.py
withub.identity
util_download.py
withub.download
andub.grabdata
- The
__init__
imports are now statically generated, this fixes the random third party attributes (e.g.expanduser
,Thread
) that were exposed in the__init__
file. ProgIter
now uses scientific notation when it is smallub.AutoOrderedDict
now inherits fromub.AutoDict
- tests are now running using
pytest
andxdoctest
ub.cmd
now uses thread based logging
- Fixed many failing tests on windows
- Small bug and documentation fixes.
ub.cmd
does not work correctly on windows- some Unicode formatting does not work correctly on windows
ub.repr2
andub.hzcat
ub.color_text
- Add
ub.argflag
andub.argval
ub.AutoDict
andub.AutoOrderedDict
.- Many undocumented changed
- Starting a changelog
- First release of ubelt
- Changed from and before this time are undocumented