Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broker concurrent limit is invalid #396

Open
LL-Ling opened this issue Dec 31, 2024 · 0 comments
Open

Broker concurrent limit is invalid #396

LL-Ling opened this issue Dec 31, 2024 · 0 comments

Comments

@LL-Ling
Copy link

LL-Ling commented Dec 31, 2024

Env

python: 3.12.3
os: window 10
taskiq: 0.11.10

Q

I want to limit the concurrent execution of tasks in the broker, so I set sync_tasks_pool_size=1 and max_async_tasks=1, but it didn't work as expected, and the tasks are still running all at once. I noticed that the Semaphore defined by these two options in the Receiver is not applied in callback and run_task. Is my usage incorrect, or is there a way to limit the number of concurrent tasks? Below is my code. I really like the design of TaskIQ, and I look forward to your response.

Code

import asyncio

from taskiq import InMemoryBroker
from taskiq.kicker import AsyncKicker

from taskiq_example.middleware import TestMiddleware

broker = InMemoryBroker(
sync_tasks_pool_size=1,
max_async_tasks=1
)
broker.add_middlewares(TestMiddleware())

async def best_task_ever(value: int) -> bool:
"""Solve all problems in the world."""
print("execute best_task_ever", value)
await asyncio.sleep(5)
return True

async def main():
await broker.register_task(best_task_ever).kiq(1)
await broker.register_task(best_task_ever).kiq(2)
await broker.register_task(best_task_ever).kiq(3)
await broker.register_task(best_task_ever).kiq(4)

task = broker.register_task(best_task_ever)
await AsyncKicker(
    task.task_name,
    broker,
    task.labels
).with_labels(
    func_value=best_task_ever.__name__,
    func_label="test func"
).kiq(
    value=5
)

await broker.wait_all()

if name == "main":
asyncio.run(main())

Result

It executes 5 tasks concurrently, taking about 5-6 seconds.
I need to control it so that the tasks are executed one by one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant