Skip to content

Commit

Permalink
Adds fast-paginate (#127)
Browse files Browse the repository at this point in the history
* Adds fast-paginate

---------

Co-authored-by: Deeka Wong <[email protected]>
  • Loading branch information
huangdijia and huangdijia authored Feb 18, 2023
1 parent a9114aa commit f81973b
Show file tree
Hide file tree
Showing 17 changed files with 542 additions and 2 deletions.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The most popular components for Hyperf.
|[encryption](https://github.com/friendsofhyperf/encryption)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/encryption/v)](https://packagist.org/packages/friendsofhyperf/encryption)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/encryption/downloads)](https://packagist.org/packages/friendsofhyperf/encryption)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/encryption/d/monthly)](https://packagist.org/packages/friendsofhyperf/encryption)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/encryption/d/daily)](https://packagist.org/packages/friendsofhyperf/encryption)|
|[exception-event](https://github.com/friendsofhyperf/exception-event)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/exception-event/v)](https://packagist.org/packages/friendsofhyperf/exception-event)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/exception-event/downloads)](https://packagist.org/packages/friendsofhyperf/exception-event)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/exception-event/d/monthly)](https://packagist.org/packages/friendsofhyperf/exception-event)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/exception-event/d/daily)](https://packagist.org/packages/friendsofhyperf/exception-event)|
|[facade](https://github.com/friendsofhyperf/facade)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/facade/v)](https://packagist.org/packages/friendsofhyperf/facade)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/facade/downloads)](https://packagist.org/packages/friendsofhyperf/facade)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/facade/d/monthly)](https://packagist.org/packages/friendsofhyperf/facade)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/facade/d/daily)](https://packagist.org/packages/friendsofhyperf/facade)|
|[fast-paginate](https://github.com/friendsofhyperf/fast-paginate)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/fast-paginate/v)](https://packagist.org/packages/friendsofhyperf/fast-paginate)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/fast-paginate/downloads)](https://packagist.org/packages/friendsofhyperf/fast-paginate)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/fast-paginate/d/monthly)](https://packagist.org/packages/friendsofhyperf/fast-paginate)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/fast-paginate/d/daily)](https://packagist.org/packages/friendsofhyperf/fast-paginate)|
|[grpc-validation](https://github.com/friendsofhyperf/grpc-validation)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/grpc-validation/v)](https://packagist.org/packages/friendsofhyperf/grpc-validation)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/grpc-validation/downloads)](https://packagist.org/packages/friendsofhyperf/grpc-validation)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/grpc-validation/d/monthly)](https://packagist.org/packages/friendsofhyperf/grpc-validation)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/grpc-validation/d/daily)](https://packagist.org/packages/friendsofhyperf/grpc-validation)|
|[helpers](https://github.com/friendsofhyperf/helpers)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/helpers/v)](https://packagist.org/packages/friendsofhyperf/helpers)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/helpers/downloads)](https://packagist.org/packages/friendsofhyperf/helpers)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/helpers/d/monthly)](https://packagist.org/packages/friendsofhyperf/helpers)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/helpers/d/daily)](https://packagist.org/packages/friendsofhyperf/helpers)|
|[http-client](https://github.com/friendsofhyperf/http-client)|[![Latest Stable Version](https://poser.pugx.org/friendsofhyperf/http-client/v)](https://packagist.org/packages/friendsofhyperf/http-client)|[![Total Downloads](https://poser.pugx.org/friendsofhyperf/http-client/downloads)](https://packagist.org/packages/friendsofhyperf/http-client)|[![Monthly Downloads](https://poser.pugx.org/friendsofhyperf/http-client/d/monthly)](https://packagist.org/packages/friendsofhyperf/http-client)|[![Daily Downloads](https://poser.pugx.org/friendsofhyperf/http-client/d/daily)](https://packagist.org/packages/friendsofhyperf/http-client)|
Expand Down
6 changes: 6 additions & 0 deletions composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
"hyperf/command": "~3.0.0",
"hyperf/config-center": "~3.0.0",
"hyperf/consul": "~3.0.0",
"hyperf/database": "~3.0.0",
"hyperf/db-connection": "~3.0.0",
"hyperf/di": "~3.0.0",
"hyperf/engine": "^1.3|^2.0",
Expand Down Expand Up @@ -59,6 +60,8 @@
"hyperf/etcd": "~3.0.0",
"hyperf/ide-helper": "~3.0.0",
"hyperf/nacos": "~3.0.0",
"hyperf/paginator": "~3.0.0",
"hyperf/scout": "~3.0.0",
"mockery/mockery": "^1.0",
"phpstan/phpstan": "^1.0",
"phpunit/phpunit": "^9.5",
Expand All @@ -80,6 +83,7 @@
"friendsofhyperf/encryption": "*",
"friendsofhyperf/exception-event": "*",
"friendsofhyperf/facade": "*",
"friendsofhyperf/fast-paginate": "*",
"friendsofhyperf/grpc-validation": "*",
"friendsofhyperf/helpers": "*",
"friendsofhyperf/http-client": "*",
Expand Down Expand Up @@ -122,6 +126,7 @@
"FriendsOfHyperf\\ConsoleSpinner\\": "src/console-spinner/src/",
"FriendsOfHyperf\\ExceptionEvent\\": "src/exception-event/src/",
"FriendsOfHyperf\\Facade\\": "src/facade/src/",
"FriendsOfHyperf\\FastPaginate\\": "src/fast-paginate/src/",
"FriendsOfHyperf\\GrpcValidation\\": "src/grpc-validation/src/",
"FriendsOfHyperf\\Helpers\\": "src/helpers/src/",
"FriendsOfHyperf\\HttpLogger\\": "src/http-logger/src/",
Expand Down Expand Up @@ -164,6 +169,7 @@
"FriendsOfHyperf\\ConsoleSpinner\\ConfigProvider",
"FriendsOfHyperf\\ExceptionEvent\\ConfigProvider",
"FriendsOfHyperf\\Facade\\ConfigProvider",
"FriendsOfHyperf\\FastPaginate\\ConfigProvider",
"FriendsOfHyperf\\GrpcValidation\\ConfigProvider",
"FriendsOfHyperf\\Helpers\\ConfigProvider",
"FriendsOfHyperf\\HttpLogger\\ConfigProvider",
Expand Down
4 changes: 4 additions & 0 deletions src/fast-paginate/.gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/.github export-ignore
/.vscode export-ignore
/tests export-ignore
.gitattributes export-ignore
13 changes: 13 additions & 0 deletions src/fast-paginate/.github/workflows/close-pull-request.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
name: Close Pull Request

on:
pull_request_target:
types: [opened]

jobs:
run:
runs-on: ubuntu-latest
steps:
- uses: superbrothers/close-pull-request@v3
with:
comment: "Thank you for your pull request. However, you have submitted this PR on the friendsofhyperf organization which is a read-only sub split of `friendsofhyperf/components`. Please submit your PR on the https://github.com/friendsofhyperf/components repository.<br><br>Thanks!"
25 changes: 25 additions & 0 deletions src/fast-paginate/.github/workflows/release.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
on:
push:
# Sequence of patterns matched against refs/tags
tags:
- 'v*' # Push events to matching v*, i.e. v1.0, v1.0.0

name: Release

jobs:
release:
name: Release
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Create Release
id: create_release
uses: actions/create-release@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tag_name: ${{ github.ref }}
release_name: Release ${{ github.ref }}
draft: false
prerelease: false
21 changes: 21 additions & 0 deletions src/fast-paginate/LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2022 D.J.Hwang

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
69 changes: 69 additions & 0 deletions src/fast-paginate/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Fast Paginate for Hyperf

> Fork from https://github.com/hammerstonedev/fast-paginate
## About

This is a fast `limit`/`offset` pagination macro for Hyperf. It can be used in place of the standard `paginate` methods.

This package uses a SQL method similar to a "deferred join" to achieve this speedup. A deferred join is a technique that defers access to requested columns until _after_ the `offset` and `limit` have been applied.

In our case we don't actually do a join, but rather a `where in` with a subquery. Using this technique we create a subquery that can be optimized with specific indexes for maximum speed and then use those results to fetch the full rows.

The SQL looks something like this:

```sql
select * from contacts -- The full data that you want to show your users.
where contacts.id in ( -- The "deferred join" or subquery, in our case.
select id from contacts -- The pagination, accessing as little data as possible - ID only.
limit 15 offset 150000
)
```

> You might get an error trying to run the query above! Something like `This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery.`
> In this package, we run them as [two _separate_ queries](https://github.com/hammerstonedev/fast-paginate/blob/154da286f8160a9e75e64e8025b0da682aa2ba23/src/BuilderMixin.php#L62-L79) to get around that!
The benefits can vary based on your dataset, but this method allows the database to examine as little data as possible to satisfy the user's intent.

It's unlikely that this method will ever perform worse than traditional `offset` / `limit`, although it is possible, so be
sure to test on your data!

> If you want to read 3,000 words on the theory of this package, you can head over to [aaronfrancis.com/2022/efficient-pagination-using-deferred-joins](https://aaronfrancis.com/2022/efficient-pagination-using-deferred-joins).
## Installation

This package supports Hyperf 3.0+.

To install, require the package via composer:

```shell
composer require friendsofhyperf/fast-paginate
```

There is nothing further you need to do. The service provider will be loaded automatically by Hyperf.

## Usage

Anywhere you would use `Model::query()->paginate()`, you can use `Model::query()->fastPaginate()`! That's it! The method signature is the same.

Relationships are supported as well:

```php
User::first()->posts()->fastPaginate();
```

## A Favor

If this helps you, please [start me](https://github.com/friendsofhyperf/fast-paginate) with before and after times! I'd love to know :D

Some community results so far:
* [30 seconds --> 250ms](https://twitter.com/mdavis1982/status/1482429071288066054)
* [28 seconds --> 2 seconds](https://twitter.com/joecampo/status/1483550610028957701)
* [7.5x faster](https://twitter.com/max_eckel/status/1483764319372333057)
* [1.1 seconds --> 0.1 seconds](https://twitter.com/max_eckel/status/1483852300414337032)
* [20 seconds --> 2 seconds](https://twitter.com/1ralphmorris/status/1484242437618941957)
* [2 seconds --> .2 seconds](https://twitter.com/julioelpoeta/status/1549524738980077568)

## Sponsor

If you like this project, Buy me a cup of coffee. [ [Alipay](https://hdj.me/images/alipay.jpg) | [WePay](https://hdj.me/images/wechat-pay.jpg) ]
32 changes: 32 additions & 0 deletions src/fast-paginate/composer.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"name": "friendsofhyperf/fast-paginate",
"description": "Fast paginate for Hyperf.",
"type": "library",
"license": "MIT",
"authors": [
{
"name": "Aaron Francis",
"email": "[email protected]"
},
{
"name": "huangdijia",
"email": "[email protected]"
}
],
"require": {
"hyperf/database": "~3.0.0",
"hyperf/paginator": "~3.0.0"
},
"autoload": {
"psr-4": {
"FriendsOfHyperf\\FastPaginate\\": "src/"
}
},
"extra": {
"hyperf": {
"config": [
"FriendsOfHyperf\\FastPaginate\\ConfigProvider"
]
}
}
}
24 changes: 24 additions & 0 deletions src/fast-paginate/src/BuilderMixin.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
<?php

declare(strict_types=1);
/**
* This file is part of friendsofhyperf/components.
*
* @link https://github.com/friendsofhyperf/components
* @document https://github.com/friendsofhyperf/components/blob/3.x/README.md
* @contact [email protected]
*/
namespace FriendsOfHyperf\FastPaginate;

class BuilderMixin
{
public function simpleFastPaginate()
{
return (new FastPaginate())->simpleFastPaginate();
}

public function fastPaginate()
{
return (new FastPaginate())->fastPaginate();
}
}
23 changes: 23 additions & 0 deletions src/fast-paginate/src/ConfigProvider.php
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<?php

declare(strict_types=1);
/**
* This file is part of friendsofhyperf/components.
*
* @link https://github.com/friendsofhyperf/components
* @document https://github.com/friendsofhyperf/components/blob/3.x/README.md
* @contact [email protected]
*/
namespace FriendsOfHyperf\FastPaginate;

class ConfigProvider
{
public function __invoke(): array
{
return [
'listeners' => [
Listener\RegisterMixinListener::class,
],
];
}
}
Loading

0 comments on commit f81973b

Please sign in to comment.