Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] - Integration with msgspec. #204

Merged
merged 11 commits into from
Nov 2, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ use it.

## Key Features

* **Fluid and Fast**: Thanks to Starlette and Pydantic.
* **Fluid and Fast**: Thanks to Starlette and Pydantic/msgpec.
* **Fast to develop**: Thanks to the simplicity of design, the development times can be reduced exponentially.
* **Intuitive**: If you are used to the other frameworks, Esmerald is a no brainer to develop.
* **Easy**: Developed with design in mind and easy learning.
Expand All @@ -166,7 +166,7 @@ distribute them.
* **Dependency Injection**: Like any other great framework out there.
* **Simplicity from settings**: Yes, we have a way to make the code even cleaner by introducing settings
based systems.
* **Database**: Out of the box support for async databases.
* **msgspec** - Support for `msgspec`.

## Relation to Starlette and other frameworks

Expand Down
3 changes: 2 additions & 1 deletion docs/esmerald.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ example.

## Key Features

* **Fluid and Fast**: Thanks to Starlette and Pydantic.
* **Fluid and Fast**: Thanks to Starlette and Pydantic/msgpec.
* **Fast to develop**: Thanks to the simplicity of design, the development times can be reduced exponentially.
* **Intuitive**: If you are used to the other frameworks, Esmerald is a no brainer to develop.
* **Easy**: Developed with design in mind and easy learning.
Expand All @@ -172,6 +172,7 @@ distribute them.
* **Scheduler**: Yes, that's right, we come with a scheduler for those background tasks.
* **Simplicity from settings**: Yes, we have a way to make the code even cleaner by introducing settings
based systems.
* **msgspec** - Support for `msgspec`.

## Relation to Starlette and other frameworks

Expand Down
262 changes: 262 additions & 0 deletions docs/msgspec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
# MsgSpec

Prior to version 2.4.0, as well known, Esmerald was using Pydantic to make the life easier
for almost everyone using the framework but things evolved and it was necessary to support
other ways to validate data too, [msgspec](https://jcristharif.com/msgspec/) became predominant
and widely adopted by the community and therefore it made sense also to provide support in Esmerald.

```python
from esmerald.datastructures.msgspec import Struct
```

!!! Warning
For a full integration of `Esmerald` with [OpenAPI](./openapi.md) this is **mandatory** to be
used. Using `msgspec.Struct` for OpenAPI models will incur in errors.

`esmerald.datastructures.msgspec.Struct` **is exactly the same** as `msgspec.Struct` but with
extras added for [OpenAPI](./openapi.md) purposes.

## What is `msgspec`

As their documentation mention:

> msgspec is a fast serialization and validation library, with builtin support for JSON, MessagePack, YAML, and TOML.

`msgspec` and `Pydantic` are two extremely powerful libraries and both serve also different purposes
but there are a lot of people that prefer `msgspec` to Pydantic for its performance.

A good example, as per `msgspec` documentation.

```python
import msgspec

class User(msgspec.Struct):
"""A new type describing a User"""
name: str
groups: set[str] = set()
email: str | None = None
```

## `msgspec` and Esmerald

Esmerald supports `msgspec` with the nuances of what the framework can offer
**without breaking any native functionality**.

So what does this mean? Well, Esmerald for [OpenAPI documentation](./openapi.md) uses internal
processes that rely on `Pydantic` and that should remain (at least for now) but also integrates
`msgspec` in a seemless way.

This means, when you implement `msgspec` structs in your code, the error handling is delegated
to the library and no Pydantic is involved, for obvious reasons, as well as the serialisation
and deserialization of the data.

### The nuances of Esmerald

This is what Esmerald can also do for you. By nuances, what Esmerald actually means is that you
**can mix `msgspec` structs within your Pydantic models but not the other way around**.

Is this useful? Depends of what you want to do. You probably won't be using this but it is there
in case you feel like playing around.

Nothing to worry about, we will be covering this in detail.

## Importing `msgspec`

As mentioned at the very top of this document, to import the `msgspec` module you will need to:

```python
from esmerald.datastructures.msgspec import Struct
```

Now, this can be a bit confusing, right? Why we need to import from this place instead of using
directly the `msgspec.Struct`?

Well, actually you can use directly the `msgspec.Struct` but as mentioned before, **Esmerald** uses
Pydantic for the [OpenAPI](./openapi.md) documentation and to add the [nuances](#the-nuances-of-esmerald)
we all love and therefore the `esmerald.datastructures.msgspec.Struct` is simply an extended
object that adds some Pydantic flavours for the [OpenAPI](./openapi.md).

This also means that you can declare [OpenAPIResponses](./responses.md#openapi-responses) using
`msgspec`. Pretty cool, right?

!!! Warning
If you don't use the `esmerald.datastructures.msgspec.Struct`, it won't be possible
to use the `msgspec` with Pydantic. At least not in a cleaner supported way.

## How to use it

### `esmerald.datastructures.msgspec.Struct`

Well, let us see how we would work with `msgspec` inside Esmerald.

In a nutshell, it is exactly the same as you would normally do if you were creating a Pydantic
base model or a datastructure to be used within your application.

```python
{!> ../docs_src/msgspec/nutshell.py !}
```

Simple, right? Yes and there is a lot going here.

You you might have noticed, we are importing from `esmerald.datastructures.msgspec` the `Struct`
and this is a good habit to have if you care about [OpenAPI documentation](./openapi.md) and then
we are simply declaring the `data` as the `User` of type `Struct` as the data in and as a response
the `User` as well.

The reason why we declare the `User` as a response it is just to show that `msgspec`
**can also be used as another Esmerald response**.

The rest, it is still as clean as always was in Esmerald.

Now, the cool part is when we send a payload to the API, something like this:

```shell
data = {"name": "Esmerald, "email": "[email protected]"}
```

When this payload is sent, the **validations are done automatically by the `msgspec` library**
which means you can implement as many validations as you want as you would normally do while
using `msgspec`.

```python hl_lines="9-12 16-17"
{!> ../docs_src/msgspec/validations.py !}
```

And just like that, you are now using `msgspec` and all of its power within **Esmerald**.

### `msgspec.Struct`

As mentuoned before, importing from `esmerald.datastructures.msgspec.Struct` should be the way
for Esmerald to use it without any issues but you can still use the normal `msgspec.Struct` as well.

```python
{!> ../docs_src/msgspec/no_import.py !}
```

Now this is possible and it will work as normal but **it comes with limitations**. When accessing
the [OpenAPI](./openapi.md), it will raise errors.

Again, `esmerald.datatructures.msgspec.Struct` is **exactly the same** as the `msgspec.Struct` with
extra Esmerald flavours and this means you
**will not have the problem of updating the version of `msgspec` anytime you need**.

### Nested structs

Well, this is now what you can do already with `msgspec` and not directly related with Esmerald but
for example purposes, let us see how it would look like it having a nested `Struct`.

```python
{!> ../docs_src/msgspec/nested.py !}
```

One possible [payload](./extras/request-data.md#the-payload-field) would be:

```json
{
"name": "Esmerald",
"email": "[email protected]",
"address": {
"post_code": "90210",
"street_address": "California"
}
}
```

### The nuances of `Struct`

It was mentioned numerous times the use of `esmerald.datastructures.msgspec.Struct` and what it
could give you besides the obvious needed [OpenAPI](./openapi.md) documentation.

This is not the only thing. This special `datastructure` also allows you to **mix with Pydantic**
models if you want to.

Does that mean the `Struct` will be then evaluated by `Pydantic`? No, it does not.

The beauty of this system is that every `Struct`/`BaseModel` is evaluated by its own `library` which
means that if you have a `Struct` inside a `BaseModel`, the validations are done separately.

Why would you mix them if they are different? Well, in theory you wouldn't but you will never know
what people want so Esmerald offers that possibility **but not the other way around**.

Let us see how it would look like having both working side by side.

```python hl_lines="4 8 14 19"
{!> ../docs_src/msgspec/mixed.py !}
```

This works perfectly well and the payload is still like this:

```json
{
"name": "Esmerald",
"email": "[email protected]",
"address": {
"post_code": "90210",
"street_address": "California"
}
}
```

The difference is that the `address` part of the payload will be evaluated by `msgspec` and the
rest by `Pydantic`.

## Responses

As mentioned before, the `Struct` of `msgspec` can also be used as `Response` of Esmerald. This
will enable the internal mechanisms to serialize/deserialize with the power of the native library
as everyone came to love.

You can see [more details](./responses.md#other-responses) about the types of responses you can use
with Esmerald.

## OpenAPI Documentation

Now this is the nice part of `msgspec`. How can you integrate `msgspec` with [OpenAPI](./openapi.md)?

Well, as mentioned before, in the same way you would normally do. You can also use
[OpenAPIResponse](./responses.md#openapi-responses) with the `Struct` as well!

Let us see the previous example again.

```python
{!> ../docs_src/msgspec/nutshell.py !}
```

This will generate a simple OpenAPI documentation using the `Struct`.

What if you want to create `OpenAPIResponse` objects? Well, there are also three ways:

1. [As single object](#as-a-single-object).
2. [As a list](#as-a-list).
3. [Mixing with Pydantic](#mixing-with-pydantic) (the [nuance](#the-nuances-of-esmerald)).

### As a single object

```python hl_lines="27 28"
{!> ../docs_src/msgspec/openapi/single.py !}
```

### As a list

```python hl_lines="21"
{!> ../docs_src/msgspec/openapi/list.py !}
```

### Mixing with Pydantic

```python hl_lines="27"
{!> ../docs_src/msgspec/openapi/mixing.py !}
```

## Notes

This is the integration with `msgspec` and Esmerald in a simple fashion where you can take advantage
of the powerful `msgspec` library and the elegance of Esmerald.

This section covers the dos and dont's of the `Struct` and once again, use:

```python
from esmerald.datastructures.msgspec import Struct
```

And have fun!
6 changes: 6 additions & 0 deletions docs/responses.md
Original file line number Diff line number Diff line change
Expand Up @@ -419,3 +419,9 @@ Below we have a few examples of possible responses recognised by Esmerald automa
```python hl_lines="15 26 29"
{!> ../docs_src/extras/form/dataclass.py !}
```

**MsgSpec**

```python
{!> ../docs_src/responses/msgspec.py !}
```
33 changes: 33 additions & 0 deletions docs_src/msgspec/mixed.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
from typing import Union

import msgspec
from pydantic import BaseModel, EmailStr
from typing_extensions import Annotated

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct

StreetAddress = Annotated[str, msgspec.Meta(min_length=5)]
PostCode = Annotated[str, msgspec.Meta(min_length=5)]


class Address(Struct):
post_code: PostCode
street_address: Union[StreetAddress, None] = None


class User(BaseModel):
name: str
email: Union[EmailStr, None] = None
address: Address


@post()
def create(data: User) -> User:
"""
Returns the same payload sent to the API.
"""
return data


app = Esmerald(routes=[Gateway(handler=create)])
33 changes: 33 additions & 0 deletions docs_src/msgspec/nested.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
from typing import Union

import msgspec
from typing_extensions import Annotated

from esmerald import Esmerald, Gateway, post
from esmerald.datastructures.msgspec import Struct

Name = Annotated[str, msgspec.Meta(min_length=5)]
Email = Annotated[str, msgspec.Meta(min_length=5, max_length=100, pattern="[^@]+@[^@]+\\.[^@]+")]
PostCode = Annotated[str, msgspec.Meta(min_length=5)]


class Address(Struct):
post_code: PostCode
street_address: Union[str, None] = None


class User(Struct):
name: Name
email: Union[Email, None] = None
address: Address


@post()
def create(data: User) -> User:
"""
Returns the same payload sent to the API.
"""
return data


app = Esmerald(routes=[Gateway(handler=create)])
21 changes: 21 additions & 0 deletions docs_src/msgspec/no_import.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
from typing import Union

import msgspec

from esmerald import Esmerald, Gateway, post


class User(msgspec.Struct):
name: str
email: Union[str, None] = None


@post()
def create(data: User) -> User:
"""
Returns the same payload sent to the API.
"""
return data


app = Esmerald(routes=[Gateway(handler=create)])
Loading