Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Issue]: Some kernel pass AB0B1 and output as std::vector<const void*> #1777

Open
Jay19751103 opened this issue Dec 26, 2024 · 2 comments
Open

Comments

@Jay19751103
Copy link

Problem Description

Hi Developer

I have question about API compatibility , for ck kernel, original FA1 use the data pointer to pass QKV and output
but for run_grouped_gemm_scale_softmax_gemm_permute.inc
I saw that the api pass std::vector<const void*>
image

Another for FA
image

When some open source use the data pointer , how could CK can cover this. For example
llama.cpp and whisper.cpp use the ggml_tensor, it does not use std::vector to store the matrix object.
It's not convenient to use this api if user want to use like the example un_grouped_gemm_scale_softmax_gemm_permute.inc, Is it reasonable ?

Operating System

Windows 11 Pro 23H2

CPU

AMD 7700X

GPU

AMD Radeon RX 7900 XT

Other

No response

ROCm Version

ROCm 6.0.0

ROCm Component

No response

Steps to Reproduce

No response

(Optional for Linux users) Output of /opt/rocm/bin/rocminfo --support

No response

Additional Information

No response

@Jay19751103 Jay19751103 changed the title [Issue]: Some kernel pass QKV and output as std::vector<const void*> [Issue]: Some kernel pass AB0B1 and output as std::vector<const void*> Dec 26, 2024
@ppanchad-amd
Copy link

Hi @Jay19751103. Internal ticket has been created to assist with your question. Thanks!

@Jay19751103
Copy link
Author

Hi @ppanchad-amd

How can I know the internal ticket number?
Also may I know that the CK support APU like which built by APU compiling GPU target myself.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants