You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have question about API compatibility , for ck kernel, original FA1 use the data pointer to pass QKV and output
but for run_grouped_gemm_scale_softmax_gemm_permute.inc
I saw that the api pass std::vector<const void*>
Another for FA
When some open source use the data pointer , how could CK can cover this. For example
llama.cpp and whisper.cpp use the ggml_tensor, it does not use std::vector to store the matrix object.
It's not convenient to use this api if user want to use like the example un_grouped_gemm_scale_softmax_gemm_permute.inc, Is it reasonable ?
Operating System
Windows 11 Pro 23H2
CPU
AMD 7700X
GPU
AMD Radeon RX 7900 XT
Other
No response
ROCm Version
ROCm 6.0.0
ROCm Component
No response
Steps to Reproduce
No response
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered:
Jay19751103
changed the title
[Issue]: Some kernel pass QKV and output as std::vector<const void*>
[Issue]: Some kernel pass AB0B1 and output as std::vector<const void*>
Dec 26, 2024
Problem Description
Hi Developer
I have question about API compatibility , for ck kernel, original FA1 use the data pointer to pass QKV and output
but for run_grouped_gemm_scale_softmax_gemm_permute.inc
I saw that the api pass std::vector<const void*>
Another for FA
When some open source use the data pointer , how could CK can cover this. For example
llama.cpp and whisper.cpp use the ggml_tensor, it does not use std::vector to store the matrix object.
It's not convenient to use this api if user want to use like the example un_grouped_gemm_scale_softmax_gemm_permute.inc, Is it reasonable ?
Operating System
Windows 11 Pro 23H2
CPU
AMD 7700X
GPU
AMD Radeon RX 7900 XT
Other
No response
ROCm Version
ROCm 6.0.0
ROCm Component
No response
Steps to Reproduce
No response
(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support
No response
Additional Information
No response
The text was updated successfully, but these errors were encountered: