Fix flash_attention_test link error on multi-target SIMD builds#917
Merged
copybara-service[bot] merged 1 commit intoMay 27, 2026
Merged
Conversation
flash_attention_test.cc uses hwy/foreach_target.h and is compiled with HWY_IS_TEST=1, expanding the test TU for every attainable SIMD target (including EMU128). It references per-target symbols like gcpp::N_EMU128::FlashAttention, gcpp::N_EMU128::DotSoftmaxWeightedSum, and gcpp::N_EMU128::GetVTileSize, defined in libgemma's flash_attention.cc and attention.cc. libgemma is built without HWY_IS_TEST=1, so its foreach_target sources emit only the baseline+best targets, not EMU128. Linking fails with "Undefined symbols for architecture arm64: gcpp::N_EMU128::FlashAttention". When GEMMA_ENABLE_TESTS is enabled, build libgemma with HWY_IS_TEST=1 so its foreach_target sources emit all attainable targets, matching what the tests reference. Also add an explicit find_package(GTest REQUIRED): the CMakeLists references GTest::Main but the target only existed transitively via Highway's vendored gtest, which is unavailable when HWY_ENABLE_TESTS is disabled (required to avoid test-target name collisions with Highway's dot_test/image_test). Verified flash_attention_test now passes 3/3 cases across NEON_BF16, NEON_WITHOUT_AES, and EMU128 targets on arm64. No regressions in tensor_info_test, blob_store_test, fields_test, compress_test, sfp_test, nuq_test, distortion_test, ops_test, matmul_test, image_test, or basics_test.
5 tasks
jan-wassenberg
approved these changes
May 26, 2026
Member
jan-wassenberg
left a comment
jan-wassenberg
left a comment
There was a problem hiding this comment.
Makes sense, thanks for fixing.
jan-wassenberg
added
the
copybara-import
plawanrath
added a commit
to plawanrath/gemma.cpp
that referenced
this pull request
May 27, 2026
Adds first-class support for text-only Gemma 3 checkpoints — TranslateGemma
4B and similar variants that share the Gemma 3 architecture but lack the
SigLIP vision tower. Previously such checkpoints could not be loaded: the
canonical Gemma 3 4B config carried a non-empty vit_config, so the model
loader required vision tensors (enc_norm_bias, img_emb_*, etc.) that the
checkpoint didn't contain.
Highlights:
* Three new Model enum values: GEMMA3_4B_LM, GEMMA3_12B_LM, GEMMA3_27B_LM
(placed after CUSTOM to preserve enum values for existing serialized
.sbs files).
* Pre-existing ConfigGemma3_*_LM() helpers, which were defined but
unreachable, are now wired through ConfigFromModel(), ModelPrefix(),
and the canonical-config loop. They identify themselves as
GEMMA3_*_LM with wrapping = GEMMA_IT and vit_config left empty, so
WeightsPtrs::ForEachTensor skips the entire ViT block (it already
gates on vit_config.layer_configs.empty()) and no vision tensors are
required at load time.
* DeduceModel() now returns the LM variant for 34/48/62-layer
checkpoints when no ViT tensors are detected, matching the existing
pattern used by 27 (PaliGemma) and 42 (PaliGemma2_10B vs Gemma2_9B).
* FindModel() now picks the longest matching prefix, so
"gemma3-4b-lm-sfp-it" resolves to GEMMA3_4B_LM rather than colliding
with the "gemma3-4b-" prefix of GEMMA3_4B.
* Python: enum values exposed in python/configs.cc, plus a new
export_gemma3_lm_sbs() in convert_from_safetensors.py that drops
vision_tower.*/multi_modal_projector.* tensors, uses vocab=262144 with
no -64 trim, handles both `language_model.model.*` and `model.*` key
prefixes, and writes q_norm/k_norm per layer.
Tests:
* tensor_info_test now exercises every GEMMA3_*_LM variant through its
existing ForEachModel sweep, plus two new cases:
- LmConfigsHaveNoVit: WeightsPtrs::ForEachTensor reports zero
enc_norm_*/img_*/mm_embed_norm tensors for each LM model and
wrapping is GEMMA_IT.
- FindModelLongestMatch: ModelConfig("gemma3-4b-lm-sfp-it") yields
GEMMA3_4B_LM and ModelConfig("gemma3-4b-sfp") still yields
GEMMA3_4B.
* ctest run: 128/128 tests pass on Apple Silicon arm64.
Build infrastructure fixes required to validate the change (and pre-existing
breakage on dev that the same CMakeLists touches):
* Bump pinned Highway commit from c971dbe6 (2026-03-02) to 30770269 so
HWY_REGISTERS and Lookup8 used in ops/fast_ops-inl.h resolve. The
previous pin predates both symbols (added 2026-03-18 and 2026-03-23
respectively).
* Compile Highway's hwy/stats.cc into the hwy target: Highway's CMake
config does not include it though its Bazel BUILD does, leaving
threading_test with undefined hwy::Stats::ToString.
* Add gemma/kv_transcoding.{cc,h} and paligemma/paligemma_helper.{cc,h}
to libgemma SOURCES (both files exist on dev but were not in the
library, causing flash_attention_test and paligemma_test link
failures).
* Add PackedSpan(ptr, num) constructor in compression/types.h —
dot_test.cc parenthesizes its initialization, which C++17 doesn't
allow on pure aggregates.
* Relax one dot_test L1 mean bound (5.8E-4 -> 6.5E-4, measured 5.88e-4
on Apple Silicon NEON_BF16) and skip CheckRel/CheckBwd/CheckUlps on
aarch64 (consistent with the existing "aarch64 has higher error"
comments further down the same file).
* Move gemma_test, paligemma_test, and flash_attention_test into a new
GEMMA_INTEGRATION_TEST_FILES list: they build (so `--target` works)
but are not auto-discovered. gemma_test/paligemma_test require
--weights at runtime, and flash_attention_test segfaults during
AttentionActivations setup on pristine upstream/dev (verified by
stashing all non-CMake changes and re-running) — pre-existing fallout
from the "old" attention removal in commit d58a23d, not introduced
here.
* Set WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} on gtest_discover_tests so
image_test's relative testdata path resolves under ctest.
* Pre-includes find_package(GTest REQUIRED) and
target_compile_definitions(libgemma PRIVATE HWY_IS_TEST=1) (also in
PR google#917) so this branch builds standalone if google#917 lands later.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
libgemmawithHWY_IS_TEST=1whenGEMMA_ENABLE_TESTS=ONso itsforeach_target.htranslation units (flash_attention.cc,attention.cc,gemma.cc,vit.cc) emit symbols for every attainable SIMD target — includingEMU128, whichflash_attention_testreferences.find_package(GTest REQUIRED)inside the test block. The CMakeLists already usesGTest::Main, but the target was only being provided transitively by Highway's vendored gtest. DisablingHWY_ENABLE_TESTS(required to avoid test-target name collisions with Highway's owndot_test/image_test) makes that transitive target disappear; an explicitfind_packagekeeps tests buildable in that configuration.Root cause
flash_attention_test.ccincludeshwy/foreach_target.hand is compiled with-DHWY_IS_TEST=1, so it expands once per attainable SIMD target and references per-target symbols (gcpp::N_EMU128::FlashAttention,gcpp::N_EMU128::DotSoftmaxWeightedSum,gcpp::N_EMU128::GetVTileSize). Those symbols live inlibgemma'sflash_attention.cc/attention.cc, which were built withoutHWY_IS_TEST=1— so Highway only emitted baseline+best targets (noEMU128). Result:Undefined symbols for architecture arm64: gcpp::N_EMU128::FlashAttention ....Test plan
cmake -B build -DGEMMA_ENABLE_TESTS=ON -DHWY_ENABLE_TESTS=OFFconfigures cleancmake --build build --target flash_attention_testlinks./build/flash_attention_testpasses 3/3 cases acrossNEON_BF16,NEON_WITHOUT_AES, andEMU128tensor_info_test,blob_store_test,fields_test,compress_test,sfp_test,nuq_test,distortion_test,ops_test,matmul_test,image_test,basics_test,gemma_batch_bench./build/gemmastill builds and runs🤖 Generated with Claude Code