Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

120000000 rows, when using ORDER BY + OFFSET will make duckdb hang for-ever : NEVER STOP, and 800% CPU #149

Open
navegador5 opened this issue Jan 19, 2025 · 0 comments

Comments

@navegador5
Copy link

sql to trigger the error:
`

await conn.all(select * from pass_vehicle ORDER BY pass_time,flowid LIMIT 1 OFFSET 2000000);

进程号 USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND
268710 root 20 0 5943028 3.9g 74548 S 880.5 6.2 4:46.17 node
进程号 USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND
268710 root 20 0 7253748 5.2g 74548 S 905.9 8.3 7:10.05 node
进程号 USER PR NI VIRT RES SHR %CPU %MEM TIME+ COMMAND
268710 root 20 0 26.7g 23.5g 74548 S 1086 37.5 25:55.07 node

`

datas:
`
total : 120000000 rows. each row has 20 column

storage using local-file:

DUCKDB# ls -l | egrep hbj_vehicle_monitor
-rw-r--r-- 1 root root 2972725248 1月 19 20:06 hbj_vehicle_monitor
-rw-r--r-- 1 root root 3805 1月 19 20:06 hbj_vehicle_monitor.wal

{
column_name: 'flowid',
column_type: 'VARCHAR',
null: 'YES',
key: null,
default: null,
extra: null
}
{
column_name: 'pass_time',
column_type: 'TIMESTAMP',
null: 'YES',
key: null,
default: null,
extra: null
}

`

nodejs:
Node.js v22.13.0.

machine env:
`

cat /proc/cpuinfo | egrep "processor|model name"

processor : 0
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 1
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 2
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 3
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 4
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 5
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 6
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 7
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 8
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 9
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 10
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 11
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 12
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 13
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 14
model name : 12th Gen Intel(R) Core(TM) i9-12900K
processor : 15
model name : 12th Gen Intel(R) Core(TM) i9-12900K

DUCKDB# cat /proc/meminfo | egrep "Hugepagesize|Mem"
MemTotal: 65849100 kB
MemFree: 3627328 kB
MemAvailable: 35407524 kB
Hugepagesize: 2048 kB

DUCKDB# uname -a
Linux debian 5.10.0-32-amd64 #1 SMP Debian 5.10.223-1 (2024-08-10) x86_64 GNU/Linux

`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant