Change lakeFS package to lakefs-sdk in place of lakefs-client
#107
Conversation
|
So I assume the errors mean that I now need to |
Yup, seems so. |
|
This is an upstream issue, tracked in treeverse/lakeFS#6795. |
nicholasjng
force-pushed
the
lakefs-sdk
branch
from
edf82da to
d05c07a
Compare
In preparation for the lakeFS v1 launch, change the package to depend on their newly launched `lakefs-sdk` to keep up to date with their upstream changes.
This is largely a name replacement only, with the exception of the model imports now happening from the `lakefs_sdk.models` subpackage.
Mostly a name-only replacement.
nicholasjng
force-pushed
the
lakefs-sdk
branch
from
d05c07a to
6c71acb
Compare
Otherwise we end up with a dependency conflict in CI.
This change comes from the move to pydantic, which requires keyword arguments for its initializations.
This was changed in the lakeFS SDK from the buffered reader used before. Now, we need to fetch byte ranges, which we were doing in the `LakeFSFile` already, with every `objects_api.get_object` call. The main difference, however, is that we cannot just pass increments of the file system's chunk size, because that will create HTTP416 errors (request range not satisfiable). Instead, first fetch the object size from the backend via `fs.info()`, then guard against the overflow by max'ing it with the next increment of the chunksize. We also change a test to purposefully require multiple `get_object` range requests to see if the logic works as intended.
Codecov ReportAttention:
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## main #107 +/- ##
==========================================
- Coverage 88.84% 87.15% -1.69%
==========================================
Files 7 7
Lines 466 467 +1
Branches 61 62 +1
==========================================
- Hits 414 407 -7
- Misses 29 35 +6
- Partials 23 25 +2
☔ View full report in Codecov by Sentry. |
|
The demo notebook still imports from the old Edit: same goes for the readme and uses cases documentation |
Command used: nbstripout --extra-keys cell.attachments demos/rain_prediction.ipynb
In preparation for the lakeFS v1 launch, change the package to depend on their newly launched
lakefs-sdkto keep up to date with their upstream changes.The import change is largely a name replacement only, with the exception of the model imports now happening from the
lakefs_sdk.modelssubpackage.