Oriel — AI & technology worth noticing, every day.

One Word, Two GPUs: Why Gemma 4 Answers the Same Question Differently on a Laptop and a Tesla T4

Post · https://dev.to/gde/one-word-two-gpus-why-gemma-4-answers-the-same-question-differently-on-a-laptop-and-a-tesla-t4-1dj2

Same GGUF file, same llama.cpp build, greedy decoding, and a GTX 1650 Ti answers $20 where a Tesla T4 answers 2000. One GSM8K question followed down to the word where the two cards part, the order in which each card adds its numbers, and the one stock setting that makes them agree.

First seen by Oriel on .

AI summary

This investigation shows why Gemma 4 E2B produces different deterministic answers on a GTX 1650 Ti and Tesla T4, tracing divergence to GPU-specific floating-point accumulation and cuBLAS behavior.

  • The GTX 1650 Ti answered GSM8K test-358 with $20, while the Tesla T4 answered 2000 by calculating cents correctly but violating the grader’s dollar format.
  • llama.cpp splits matrix work across 16 versus 40 streaming multiprocessors, changing float32 summation order; one BF16 matrix additionally uses card-dependent cuBLAS methods.
  • The cards matched for 124 tokens before diverging between “ spent” and “ saved”; 8,960-number simulations produced sums of 47.0389595 and 47.0389099.
  • Using both fixes produced identical answers and token scores across 250 questions, while stock -ub 1 achieved agreement with approximately 15% higher total runtime.
  • For reproducible benchmarks, the article recommends -ub 1 and reporting the GPU, build, and settings; default serving is faster but can vary across cards.

#gemma #llama.cpp #cuda #gpu reproducibility

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Reactions

Total public reactions reported by the source; distinct from points and likes.

  • DEV Community: 28, latest observation ; +20 over observed period since

Comments

Comment count reported by the source.

  • DEV Community: 7, latest observation ; +7 over observed period since

Aliases

SourceExternal IDURL
DEV Community4977952https://dev.to/gde/one-word-two-gpus-why-gemma-4-answers-the-same-question-differently-on-a-laptop-and-a-tesla-t4-1dj2 Read discussion