可否解决并发查相同数据的问题？

[scwlkq]

比如许多台机器同时查相同的SQL （同一时间连接全被占用 阻塞其他SQL）然后全量数据再本地分片

目前的解决方式
1.提前查 放入缓存
2.分页查（要排序 性能不好）
3.串行执行 这个过程中数据会变化

理想的解决方式：xxl-job可以任务执行前多一步 准备数据，如果是相同的数据，一台机器查完分发给所有机器

如果是一个Feature的话 我愿意开发这个

For example, many machines simultaneously query the same SQL (all connections are occupied at the same time, blocking other SQL), and then the entire data is locally sharded
The current solution

1. Check and cache in advance
2. Page lookup (due to poor sorting performance)
   3.During the serial execution of this process, the data will change

Ideal solution: xxl job can prepare additional data before task execution. If the data is the same, one machine can check it and distribute it to all machines

If it were a feature, I would be willing to develop it

[scwlkq]

（目前的解决方案是：每个机器取随机数 sleep一会查）

[scwlkq]

Hello, we tried to solve the issue.

This is what we did:

Implement a new feature in xxl-job to allow data preparation before task execution. This will enable one executor to query and prepare data, then distribute it to all executors for processing. This addresses the issue of multiple machines querying the same large dataset simultaneously.

You can review changes in this commit: trinks-slam818@736b88a.

Caution

Disclaimer: The concept of solution was created by AI and you should never copy paste this code before you check the correctness of generated code. Solution might not be complete, you should use this code as an inspiration only.

Latta AI seeks to solve problems in open source projects as part of its mission to support developers around the world. Learn more about our mission at https://latta.ai/ourmission . If you no longer want Latta AI to attempt solving issues on your repository, you can block this account.

nb