Repository navigation
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
github-actions
Bot
added
size/L
| grad_query, grad_key, grad_value = torch.empty_like(query), torch.empty_like(key), torch.empty_like(value) | ||
|
|
||
| _ = wrapped_backward_fn( | ||
| grad_out, query, key, value, out = ( |
There was a problem hiding this comment.
I think the grad_* tensors would need to be contiguous as well right?
github-actions
Bot
added
size/M
|
note this pr is a work in progress - and may require some small changes upstream to the flash attn2 to improve the exposure of public apis. will add updates here as this work progresses |
cc4003b to
e463787
Compare
github-actions
Bot
added
size/S
drbh
changed the title
|
this pr has changed to simply bump the version to the latest (future) version of fa2, relevant pr here huggingface/kernels-community#1226 once the new version is published, no changes should be needed on the diffuser side as the apis needed have been promoted to be public/available |
0afaa6d to
e860001
Compare
|
@drbh Okay to close this since the fix has been made upstream? |
|
hi @DN6 please feel free to close/review this pr based on what makes most sense for this project. I'd recommend using the latest version of the kernel (v4) as it has some minor performance improvements, adds back dropout support and aligns with the non kernels flash attention api (more details in the kernels-community pr description here huggingface/kernels-community#1226 (comment)) |
this pr update the attention dispatch to avoid using private flash attention apis that have been removed in all hub flash attention versions >1