Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Draft][Please review] Make Paginator use simpler queries when there are no sql joins used (#8278) #11595

Draft
wants to merge 10 commits into
base: 3.3.x
Choose a base branch
from
93 changes: 84 additions & 9 deletions src/Tools/Pagination/Paginator.php
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
use Doctrine\ORM\Internal\SQLResultCasing;
use Doctrine\ORM\NoResultException;
use Doctrine\ORM\Query;
use Doctrine\ORM\Query\Expr\Join;
use Doctrine\ORM\Query\Parameter;
use Doctrine\ORM\Query\Parser;
use Doctrine\ORM\Query\ResultSetMapping;
Expand All @@ -21,6 +22,7 @@
use function array_map;
use function array_sum;
use function assert;
use function count;
use function is_string;

/**
Expand All @@ -38,14 +40,43 @@
private readonly Query $query;
private bool|null $useOutputWalkers = null;
private int|null $count = null;
private bool $fetchJoinCollection;
/**
* @var bool The auto-detection of queries style was added a lot later to this class, and this
d-ph marked this conversation as resolved.
Show resolved Hide resolved
* class historically was by default using the more complex queries style, which means that
* the simple queries style is potentially very under-tested in production systems. The purpose
* of this variable is to not introduce breaking changes until an impression is developed that
* the simple queries style has been battle-tested enough.
*/
private bool $queryStyleAutoDetectionEnabled = false;
/** @var bool|null Null means "undetermined". */
private bool|null $queryHasHavingClause = null;

/** @param bool $fetchJoinCollection Whether the query joins a collection (true by default). */
/**
* @param bool|null $fetchJoinCollection Whether the query joins a collection (true by default). Set
* to null to enable auto-detection of this parameter, however note that the auto-detection requires
* a QueryBuilder to be provided to Paginator, otherwise this parameter goes back to its default value.
* Also, for now, when the auto-detection of this parameter is enabled, then auto-detection
* of $useOutputWalkers and of the CountWalker::HINT_DISTINCT is also enabled.
*/
public function __construct(
Query|QueryBuilder $query,
private readonly bool $fetchJoinCollection = true,
bool|null $fetchJoinCollection = true,
d-ph marked this conversation as resolved.
Show resolved Hide resolved
) {
if ($fetchJoinCollection === null) {
$fetchJoinCollection = $this->autoDetectFetchJoinCollection($query);

Check warning on line 67 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L67

Added line #L67 was not covered by tests
}

if ($fetchJoinCollection === null) {
$this->fetchJoinCollection = true;

Check warning on line 71 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L71

Added line #L71 was not covered by tests
} else {
$this->fetchJoinCollection = $fetchJoinCollection;
$this->queryStyleAutoDetectionEnabled = true;
}

if ($query instanceof QueryBuilder) {
$query = $query->getQuery();
$this->queryHasHavingClause = $query->getDQLPart('having') !== null;
$query = $query->getQuery();
}

$this->query = $query;
Expand Down Expand Up @@ -154,6 +185,23 @@
return new ArrayIterator($result);
}

/** @return bool|null Null means that auto-detection could not be carried out. */
private function autoDetectFetchJoinCollection(Query|QueryBuilder $query): bool|null

Check warning on line 189 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L189

Added line #L189 was not covered by tests
{
// For now, only working with QueryBuilder is supported.
if (! $query instanceof QueryBuilder) {
return null;

Check warning on line 193 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L192-L193

Added lines #L192 - L193 were not covered by tests
}

/** @var array<string, Join[]> $joinsPerRootAlias */
$joinsPerRootAlias = $query->getDQLPart('join');

Check warning on line 197 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L197

Added line #L197 was not covered by tests

// For now, do not try to investigate what kind of joins are used. It is, however, doable
// to detect a presence of only *ToOne joins via the access to joined entity classes'
// metadata (see: QueryBuilder::getEntityManager()->getClassMetadata(className)).
return count($joinsPerRootAlias) > 0;

Check warning on line 202 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L202

Added line #L202 was not covered by tests
}

private function cloneQuery(Query $query): Query
{
$cloneQuery = clone $query;
Expand All @@ -171,13 +219,30 @@
/**
* Determines whether to use an output walker for the query.
*/
private function useOutputWalker(Query $query): bool
private function useOutputWalker(Query $query, bool $forCountQuery = false): bool
{
if ($this->useOutputWalkers === null) {
return (bool) $query->getHint(Query::HINT_CUSTOM_OUTPUT_WALKER) === false;
if ($this->useOutputWalkers !== null) {
return $this->useOutputWalkers;
}

return $this->useOutputWalkers;
// When a custom output walker already present, then do not use the Paginator's.
if ($query->getHint(Query::HINT_CUSTOM_OUTPUT_WALKER) !== false) {
return false;
}

// When not joining onto *ToMany relations, then do not use the more complex CountOutputWalker.
// phpcs:ignore SlevomatCodingStandard.ControlStructures.UselessIfConditionWithReturn.UselessIfCondition
d-ph marked this conversation as resolved.
Show resolved Hide resolved
if (
$forCountQuery
d-ph marked this conversation as resolved.
Show resolved Hide resolved
&& $this->queryStyleAutoDetectionEnabled
&& ! $this->fetchJoinCollection === false
d-ph marked this conversation as resolved.
Show resolved Hide resolved
// CountWalker doesn't support the "having" clause, while CountOutputWalker does.
&& $this->queryHasHavingClause === false
) {
return false;

Check warning on line 242 in src/Tools/Pagination/Paginator.php

View check run for this annotation

Codecov / codecov/patch

src/Tools/Pagination/Paginator.php#L242

Added line #L242 was not covered by tests
}

return true;
}

/**
Expand Down Expand Up @@ -205,10 +270,20 @@
$countQuery = $this->cloneQuery($this->query);

if (! $countQuery->hasHint(CountWalker::HINT_DISTINCT)) {
$countQuery->setHint(CountWalker::HINT_DISTINCT, true);
$hintDistinctDefaultTrue = true;

// When not joining onto *ToMany relations, then use a simpler COUNT query in the CountWalker.
d-ph marked this conversation as resolved.
Show resolved Hide resolved
if (
$this->queryStyleAutoDetectionEnabled
&& $this->fetchJoinCollection === false
d-ph marked this conversation as resolved.
Show resolved Hide resolved
) {
$hintDistinctDefaultTrue = false;
}

$countQuery->setHint(CountWalker::HINT_DISTINCT, $hintDistinctDefaultTrue);
}

if ($this->useOutputWalker($countQuery)) {
if ($this->useOutputWalker($countQuery, forCountQuery: true)) {
$platform = $countQuery->getEntityManager()->getConnection()->getDatabasePlatform(); // law of demeter win

$rsm = new ResultSetMapping();
Expand Down
Loading