Change RMSNorm eps to 1e-6 (#173)

I think this is what is used in the Python code
pull/109/head^2 master-22213a1
Georgi Gerganov 1 year ago
parent d7def1a752
commit 22213a17b5
No known key found for this signature in database
GPG Key ID: 449E073F9DC10735

@ -5556,7 +5556,7 @@ static void ggml_compute_forward_rms_norm_f32(
const size_t nb2 = dst->nb[2];
const size_t nb3 = dst->nb[3];
const ggml_float eps = 1e-5f; // TODO: make this a parameter
const ggml_float eps = 1e-6f; // TODO: make this a parameter
// TODO: optimize
for (int i03 = 0; i03 < ne03; i03++) {

Loading…
Cancel
Save