Skip to content

Commit

Permalink
attach ShellCommandResult to ShellCommandError
Browse files Browse the repository at this point in the history
  • Loading branch information
adamhl8 committed Mar 20, 2023
1 parent 2b787cc commit 56e2575
Show file tree
Hide file tree
Showing 5 changed files with 70 additions and 44 deletions.
69 changes: 38 additions & 31 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@
- [Why?](#why)
- [Similar Projects](#similar-projects)
- [Advanced Usage](#advanced-usage)
- [Multiple Commands](#multiple-commands)
- [Shell Command Result](#shell-command-result)
- [Exception Handling](#exception-handling)
- [Multiple Commands / Persisting Environment](#multiple-commands--persisting-environment)
- [Options](#options)
- [Output](#output)
- [Environment Variables](#environment-variables)
Expand All @@ -41,33 +43,34 @@ X("echo hello world")
Easily get a command's output, do something with it, and run another command using the value:

```python
result = X("echo hello world | sed 's/world/there/'").out
greeting = result.capitalize()
output = X("echo hello world | sed 's/world/there/'").out
greeting = output.capitalize()
X(f"echo 'echo {greeting}' >> .bashrc")
```

An exception is raised if a command exits with a non-zero status (like bash's `set -e`):

```python
X("curl https://invalid.url -o ~/super_important.tar.gz") # curl exits with exit status of 6
text = X("grep hello /non/existent/file").out # grep exits with a non-zero status
# ^ Raises ShellCommandError so the rest of the script doesn't run
X("tar -vxzf ~/super_important.tar.gz")
my_text_processor(text)
```

Or, maybe you want to handle the error:

```python
text = ""
try:
X("curl https://invalid.url -o ~/super_important.tar.gz")
text = X("grep hello /non/existent/file").out
except ShellCommandError:
X("curl https://definitely-valid.url -o ~/super_important.tar.gz")
X("tar -vxzf ~/super_important.tar.gz")
text = X("grep hello /file/that/definitely/exists").out
my_text_processor(text)
```

Pipeline errors are not masked (like bash's `set -o pipefail`):

```python
X("grep hello /non/existent/file | tee new_file")
X("grep hello /non/existent/file | tee new_file") # tee gets nothing from grep, creates an empty file, and exits with status 0
# ^ Raises ShellCommandError
```

Expand Down Expand Up @@ -96,15 +99,17 @@ A note on compatability: ShellRunner should work with on any POSIX-compliant sys

Confirmed compatible with `sh` (dash), `bash`, `zsh`, and `fish`.

Commands are automatically run with the shell that invoked your python script (this can be overridden):
Commands are automatically run with the shell that invoked your python script (this can be [overridden](#options)):

```python
# my_script.py
X("echo hello | string match hello")
# Works if my_script.py is executed under fish. Will obviously fail if using bash.
```

`X` returns a `NamedTuple` containing the output of the command and a list of its exit status(es), accessed via `.out` and `.status` respectively.
### Shell Command Result

`X` returns a `ShellCommandResult` (`NamedTuple`) containing the output of the command and a list of its exit status(es), accessed via `.out` and `.status` respectively.

```python
result = X("echo hello")
Expand All @@ -130,45 +135,47 @@ status = X("grep hello /non/existent/file | tee new_file").status
# if invoked with sh: No exception is raised and status = [0]
```

### Multiple Commands
### Exception Handling

Sometimes you might want to do something like this:
`ShellCommandError` also receives the information from the failed command, which means you can do something like this:

```python
# Pretend current working directory is ~/
X("curl https://definitely-valid.url -o /tmp/super_important.tar.gz")
X("cd /tmp/")
X("tar -vxzf super_important.tar.gz")
# ^ Raises exception because tar cannot find the file
try:
X("echo hello && false") # Pretend this is some command that prints something but exits with a non-zero status
except ShellCommandError as e:
print(f'Command failed. Got output "{e.out}" with exit status {e.status}')
```

This fails because each call of `X` invokes a new instance of the shell, so things like `cd` don't persist.
### Multiple Commands / Persisting Environment

Each call of `X` invokes a new instance of the shell, so things like environment variables or directory changes don't persist.

Sometimes you might want to do something like this:

```python
X("MY_VAR=hello")
X("grep $MY_VAR /file/that/exists") # MY_VAR doesn't exist
# ^ Raises ShellCommandError
```

A (bad) solution would be to do this:

```python
X("""
curl https://definitely-valid.url -o /tmp/super_important.tar.gz
cd /tmp/
tar -vxzf super_important.tar.gz
""")
X("MY_VAR=hello; grep $MY_VAR /file/that/exists")
```

However, this sort of defeats the purpose of ShellRunner because that would be run as one command, so no error handling can take place.
This sort of defeats the purpose of ShellRunner because that would be run as one command, so no error handling can take place on commands before the last one.

Instead, `X` also accepts a list of commands:
Instead, `X` also accepts a list of commands where each command is run in the same shell instance and goes through the normal error handling:

```python
X([
"curl https://definitely-valid.url -o /tmp/super_important.tar.gz",
"cd /tmp/",
"tar -vxzf super_important.tar.gz"
"MY_VAR=hello",
"grep $MY_VAR /file/that/exists",
])
# Works!
```

Each command is run in the same shell instance and goes through the normal error checking.

## Options

There are a few keyword arguments you can provide to adjust the behavior of `X`:
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[project]
name = "python-shellrunner"
version = "0.1.2"
version = "0.2.0"
description = "Write safe shell scripts in Python."
authors = [
{name = "adamhl8", email = "[email protected]"},
Expand Down
15 changes: 9 additions & 6 deletions src/helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,19 +4,22 @@
from typing import NamedTuple, TypeVar


class ShellCommandResult(NamedTuple):
out: str
status: list[int]


class ShellCommandError(ChildProcessError):
pass
def __init__(self, message: str, result: ShellCommandResult):
super().__init__(message)
self.out = result.out
self.status = result.status


class EnvironmentVariableError(ValueError):
pass


class ResultTuple(NamedTuple):
out: str
status: list[int]


# Returns the full path of parent process/shell. That way commands are executed using the same shell that invoked this script.
def get_parent_shell_path() -> Path:
try:
Expand Down
18 changes: 13 additions & 5 deletions src/shellrunner.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,14 @@
import sys
from inspect import cleandoc

from helpers import Env, ResultTuple, ShellCommandError, get_parent_shell_path, resolve_option, resolve_shell_path
from helpers import (
Env,
ShellCommandError,
ShellCommandResult,
get_parent_shell_path,
resolve_option,
resolve_shell_path,
)

# We only need to do this once (on import) since it should never change between calls of X.
parent_shell_path = get_parent_shell_path()
Expand All @@ -23,7 +30,7 @@ def X( # noqa: N802
check: bool | None = None,
show_output: bool | None = None,
show_commands: bool | None = None,
) -> ResultTuple:
) -> ShellCommandResult:
# We default each argument to None rather than the "real" defaults so we can detect if the user actually passed something in.
# Passed in arguments take precedence over the related environment variable.
shell = resolve_option(shell, Env.get_str("SHELLRUNNER_SHELL"), default="")
Expand Down Expand Up @@ -141,11 +148,12 @@ def X( # noqa: N802
message = "Something went wrong. Failed to capture an exit status."
raise RuntimeError(message)

result = ShellCommandResult(output.rstrip(), status_list)

if check:
for status in status_list:
if status != 0:
# TODO: attach the result to the exception
message = f"Command exited with non-zero status: {status_list}"
raise ShellCommandError(message)
raise ShellCommandError(message, result)

return ResultTuple(output.rstrip(), status_list)
return result
10 changes: 9 additions & 1 deletion src/test_shellrunner.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ def get_parent_shell_info():
shell_path = get_parent_shell_path()

# If pytest is invoked via pdm, the parent process will be python, not the shell, so we fallback to bash.
if shell_path.name.startswith("python"):
# Similarly, if pytest is invoked via VS Code, the parent process will be node.
if shell_path.name.startswith("python") or shell_path.name.startswith("node"):
return ShellInfo("/bin/bash", "bash")
return ShellInfo(f"{shell_path}", shell_path.name)

Expand Down Expand Up @@ -92,6 +93,13 @@ def test_pipeline_with_unknown_command_raises_error(self, shell: str, shell_comm
X("true | foo", shell=shell)
assert str(cm.value).startswith(shell_command_error_message)

def test_shellcommanderror_has_result(self, shell: str, shell_command_error_message: str):
with pytest.raises(ShellCommandError) as cm:
X("echo test && false", shell=shell)
assert str(cm.value).startswith(shell_command_error_message)
assert cm.value.out == "test"
assert cm.value.status == [1]

def test_command_list(self, shell: str):
result = X(["echo test", "echo test"], shell=shell)
assert result.out == "test\ntest"
Expand Down

0 comments on commit 56e2575

Please sign in to comment.