🚀 By Science and FOR SCIENCE 🇫🇷

PhysiQuanty PRO

PhysiQuanty

·

AI & ML interests

Theoretical Physics, Invariant Tokenization, Standard Model of Particle Physics Applied ML 🇫🇷

Recent Activity

upvoted an article about 16 hours ago

🔁 Teaching a 15M French LLM to think deeper — and to know when to stop 🇫🇷

reacted to RDTvlokip's post with 🚀 about 16 hours ago

I finally changed the architecture of my 15M French LLM. It worked. Then I almost fooled myself about how much and catching that was the real win. After proving last time that architecture is a threshold, not a lever, I got stubborn: could I change how the model learns? Four honest attempts, Lion, a sharper AdamW β2, multi-token prediction, LayerScale. Four failures. The bottleneck wasn't the learning rule either. So I changed the shape of the computation instead: loop the same transformer blocks 4×, deeper reasoning, zero added parameters. It beat the baseline on perplexity, the first thing in the whole project to move that number. Then I added my own twist: let each token decide how deep to think, halting on its own entropy. My first evaluation was spectacular. Coherence up 65%. Hallucinated names down 62%. It was noise. Eight prompts, one seed. I re-ran on 50 prompts × 200 tokens and watched the gains shrink to "modest" and on out-of-domain prompts, recurrence actually made things worse. No universal winner. And none of it is new: it's Adaptive Computation Time (2016), the Universal Transformer (2018), and LoopViT (2026), recombined and measured honestly. The real lesson: A number from 8 prompts is a rumor. The eval harness that kills your own best result is worth more than the result it kills. Cite your lineage. Stay preliminary until multiple seeds say otherwise. The three models are live. The write-up is honest about every caveat 👇 🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-15m-french-llm-to-think-deeper

reacted to RDTvlokip's post with 🔥 about 16 hours ago

I finally changed the architecture of my 15M French LLM. It worked. Then I almost fooled myself about how much and catching that was the real win. After proving last time that architecture is a threshold, not a lever, I got stubborn: could I change how the model learns? Four honest attempts, Lion, a sharper AdamW β2, multi-token prediction, LayerScale. Four failures. The bottleneck wasn't the learning rule either. So I changed the shape of the computation instead: loop the same transformer blocks 4×, deeper reasoning, zero added parameters. It beat the baseline on perplexity, the first thing in the whole project to move that number. Then I added my own twist: let each token decide how deep to think, halting on its own entropy. My first evaluation was spectacular. Coherence up 65%. Hallucinated names down 62%. It was noise. Eight prompts, one seed. I re-ran on 50 prompts × 200 tokens and watched the gains shrink to "modest" and on out-of-domain prompts, recurrence actually made things worse. No universal winner. And none of it is new: it's Adaptive Computation Time (2016), the Universal Transformer (2018), and LoopViT (2026), recombined and measured honestly. The real lesson: A number from 8 prompts is a rumor. The eval harness that kills your own best result is worth more than the result it kills. Cite your lineage. Stay preliminary until multiple seeds say otherwise. The three models are live. The write-up is honest about every caveat 👇 🔗 https://huggingface.co/blog/RDTvlokip/teaching-a-15m-french-llm-to-think-deeper

View all activity

Organizations

PhysiQuanty 's models 13

PhysiQuanty/Self-Predicting-Gradient-Descent

Updated 4 days ago

PhysiQuanty/Binary-LLM-POC

Text Generation • 10.7M • Updated May 2 • 23 • 10

PhysiQuanty/Wiki-Test2

79.7M • Updated Mar 4 • 4

PhysiQuanty/Wiki-Test

79.7M • Updated Mar 3 • 2 • 1

PhysiQuanty/Patenty-Test2-Radix-65536

79.7M • Updated Mar 2 • 2

PhysiQuanty/VOCAB-4294967296-FOUR-LOGITS-256

13.7M • Updated Feb 25 • 2 • 2

PhysiQuanty/Patent-Dual-Cross-Entropie

46.5M • Updated Feb 24 • 3

PhysiQuanty/Patent-Test-Radix-65536-AutoTokenizer_FineTune

79.7M • Updated Feb 23 • 3

PhysiQuanty/Patenty-0.1

79.7M • Updated Feb 22 • 2

PhysiQuanty/Patenty1-0.1B

79.7M • Updated Feb 19 • 2

PhysiQuanty/Patent-Test-Radix-65536-AutoTokenizer

79.7M • Updated Feb 13 • 1

PhysiQuanty/Patent-Test-Radix-65536

79.7M • Updated Feb 12 • 2

PhysiQuanty/Binary-Addition-LLM-POC

Text Generation • 10.7M • Updated Feb 5 • 7 • 3