Toy Model cosine similarities

True vs True

gt_feature_cosine.png

Vanilla decoder vs True

vanilla_decoder_gt_cosine.png

Matryoshka decoder vs True

matryoshka_decoder_gt_cosine.png

Vanilla encoder vs True

vanilla_encoder_gt_cosine.png

Matryoshka encoder vs True

matryoshka_encoder_gt_cosine.png

L0/FVU comparisons

Make sure to zoom in on small values, the graphs are interactive.

https://sparselatents.com/l0_fvu/Ra.html

https://sparselatents.com/l0_fvu/A.html