Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add some documentation on errors #508

Merged
merged 4 commits into from
Nov 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions docs/errors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Error Handling

Sometimes things go wrong in a ServiceX query, due to a badly-formed query, unavailable input files, server error, or something else. Issues can arise either before submission, during submission, after submission, or when retrieving results.

## Errors before submission
* If the request does not parse properly, a `ValidationError` will be raised and nothing will be submitted.
* If the request includes invalid values, an appropriate exception (`ValueError`, `RuntimeError`, `TypeError`, `NameError`) will be raised and nothing will be submitted.
* If the authentication information in your `.servicex`/`servicex.yaml` file is incorrect, an `AuthorizationError` will be raised and nothing will be submitted.

## Errors during or after submission
The results of a `deliver` call are returned in a dictionary that maps the names of the various samples in the request to a list of file names or URLs. Technically this result is a type called `GuardList`: it behaves like a normal list if everything was successful with that particular sample, but if there is an exception raised, attempting to access the list object will raise a `ReturnValueException` in your code. We do this so that you can still access successful results while marking requests that failed.

The validity of a `GuardList` can be tested by testing the return value of the `.valid()` method:

```
results = deliver(spec)
for name, resultlist in results:
if resultlist.valid():
# do something with results
else:
# handle failure
```

Of course in proper Python fashion you may prefer to handle the exception from accessing invalid data rather than test each result value (i.e., requesting forgiveness rather than asking permission). A realistic analysis script probably wants to terminate if there is an error condition in any transformation request.

If an error occurs after submission, `deliver()` will return a dict unless a severe error occurs in the client code (e.g. you interrupt your code with Ctrl-C or a Jupyter kernel interrupt). If `deliver()` throws an exception otherwise, it should be considered a bug in the code. Various errors are handled as follows:
* If a request cannot be submitted at all (for example, somehow a unparseable query is sent) then a `RuntimeError` will be raised for the corresponding sample.
* If a transformation for a specific sample is canceled or ServiceX signals a fatal error on the backend, it will raise a `ServiceXException` for that sample.
* If a transformation for a specific sample does not fully process all files, the results will depend on whether the `fail_if_incomplete` argument to `deliver()` was set to `True` (default) or `False`. If `True`, then a `ServiceXException` will be raised for that sample. If `False`, then a list of the available partial results will be returned. In either case the results will *not* be cached to the local cache database --- rerunning the code will resubmit the request. An error message will be printed with a link to a web page which summarizes errors on the server associated with the transformation. This can be caused by any runtime error: frequent causes are input files that are unavailable or errors in the query that can only be checked at run time (e.g. requesting branches that do not exist).
* If an error occurs during download (lack of disk space, permission errors, problems connecting to the remote storage, etc.) an appropriate exception will be raised for the corresponding sample.
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ re-running queries that have already been executed.
transform_request
datasets
examples
errors
yaml
command_line
contribute
Expand Down
Loading