Post
1279
Nobody knows what is doing, when you train a model, you are experimenting to advance the frontier, so keep failing ๐ซต
- 9 replies
๐ค quecto mode
appvoid
appvoid
KatyTheCutie's profile picture
beren-evans-oguz's profile picture
K-ai-Innovations's profile pictureยท
AI & ML interests
singularity through byte-level tokens and small language models
Recent Activity
reacted to Banaxi-Tech's post with ๐ 3 days ago
We're releasing Overfitter 1.0.
Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE.
It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE!
https://huggingface.co/BananaMind/Overfitter-1.0Organizations
posted an update about 23 hours ago
reacted to Banaxi-Tech's post with ๐ 3 days ago
Post
2681
We're releasing Overfitter 1.0.
Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE.
It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE!
BananaMind/Overfitter-1.0
Its a completly useless overfitted model trained on 202 epochs of SWE Bench Verified, SWE Bench Pro, Terminal Bench 2.1, DeepSWE.
It gets 100% on SWE Bench Verified, 98.6% on SWE bench Pro, 100% on Terminal Bench 2.1 and 100% on DeepSWE!
BananaMind/Overfitter-1.0
- 11 replies
replied to their post 3 days ago
Basically, they nerfed their models, then they lower limits without any transparency. Finally, they accused their users of being using their models the wrong way. Gave their free users luna by default and nerfed back to 5-hours limit plus users.
posted an update 3 days ago
Post
2551
I hope that after this OpenAI disaster on Plus users, more people start realizing why Open Weights were always the only way.
- 5 replies
replied to their post 6 days ago
A shared byte model makes the most sense when tokenization inefficiency is already high; for well-tokenized languages, the sequence expansion may still outweigh the benefits unless the SSM is efficient enough to absorb it. But even then, SSM being a real alternative is a huge deal compared to years ago.
posted an update 6 days ago
Post
988
Byte-level state-space models. That sounded pretty scary for a scientist decades ago. Now we have:
1. Knowledge that deeper layers train smoothly.
2. Knowledge that Transformers work but is quadratic on sequence length.
3. Knowledge that SSMs work even better. Numerically unstable sometimes.
4. Speculative-decoding.
5. Open high-quality data.
6. Knowledge that KD works.
It slowly feels like is no longer a bad idea.
1. Knowledge that deeper layers train smoothly.
2. Knowledge that Transformers work but is quadratic on sequence length.
3. Knowledge that SSMs work even better. Numerically unstable sometimes.
4. Speculative-decoding.
5. Open high-quality data.
6. Knowledge that KD works.
It slowly feels like is no longer a bad idea.
- 3 replies
replied to their post 8 days ago
Interesting
replied to their post 8 days ago
a little bit skeptic about 1m parameters though
Plausible on server cpus, but maybe lower for normal cpus idk
Yeah on server cpus actually makes sense
replied to their post 8 days ago
a little bit skeptic about 1m parameters though
posted an update 8 days ago
Post
158
It's 2026 and there are no instantaneous/fast vision language models for cpus yet. That's another free idea.
- 6 replies
posted an update 10 days ago
Post
135
CaraArchive highlights a broader reality of putting data online: once something is publicly accessible, it becomes extremely difficult to guarantee that it will remain under your control.
If there is information or artwork that you absolutely do not want copied, archived, scraped, downloaded, or used by others, the safest option is still not to publish it publicly in the first place. That may sound obvious, but the internet was fundamentally designed to move and reproduce information, and there are countless ways to retrieve publicly accessible images:from ordinary browser tools and web scraping to automated or agentic systems.
That does not mean artists should simply accept every possible use of their work. Artists deserve meaningful control, attribution, compensation, and reasonable ways to express how their work may be used. But treating the technology and peopple using it itself as the enemy is unlikely to solve the underlying problem.
There probably isn't a technical solution that can make a publicly visible image simultaneously viewable by everyone and impossible to copy. The realistic goal should therefore be to create better norms, incentives, licensing systems, and tools around how that content is used.
Like, we can imagine a future where every artist gets his/her own credentials and some kind of fingerprint done just like blockchain works. But that requires substantial cooperation among organizations, companies and individuals.
Technology and art are not inherently opposing sides though.
If there is information or artwork that you absolutely do not want copied, archived, scraped, downloaded, or used by others, the safest option is still not to publish it publicly in the first place. That may sound obvious, but the internet was fundamentally designed to move and reproduce information, and there are countless ways to retrieve publicly accessible images:from ordinary browser tools and web scraping to automated or agentic systems.
That does not mean artists should simply accept every possible use of their work. Artists deserve meaningful control, attribution, compensation, and reasonable ways to express how their work may be used. But treating the technology and peopple using it itself as the enemy is unlikely to solve the underlying problem.
There probably isn't a technical solution that can make a publicly visible image simultaneously viewable by everyone and impossible to copy. The realistic goal should therefore be to create better norms, incentives, licensing systems, and tools around how that content is used.
Like, we can imagine a future where every artist gets his/her own credentials and some kind of fingerprint done just like blockchain works. But that requires substantial cooperation among organizations, companies and individuals.
Technology and art are not inherently opposing sides though.
reacted to mlabonne's post with โค๏ธ 14 days ago
Post
3748
Deploy local agents everywhere with LiquidAI/LFM2.5-2.6B
Unlike agents that depend on cloud APIs, local agents give you free inference, low latency, and real privacy.
Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks that burn through millions of tokens at no marginal cost!
Unlike agents that depend on cloud APIs, local agents give you free inference, low latency, and real privacy.
Removing the per-token cost changes how developers build: agents can now be massively parallelized on local hardware, running background tasks that burn through millions of tokens at no marginal cost!
- 7 replies
replied to Banaxi-Tech's post 14 days ago
Good job
reacted to Banaxi-Tech's post with ๐ 14 days ago
Post
2913
We're excited to release BananaMind 2 Pro, our final version of the Pro model.
Trained on 100B tokens it performs extremely good for its token and size class.
The training took 22 days on one RTX 5070 Ti.
Check it out at
BananaMind/BananaMind-2-Pro
We did not release a Chat version yet because it regressed. Release Later.
Follow us to know when BananaMind 2 Ultra releases and support us at
BananaMind
@Banaxi-Tech
@vovaRL
@DedeProGames
Trained on 100B tokens it performs extremely good for its token and size class.
The training took 22 days on one RTX 5070 Ti.
Check it out at
BananaMind/BananaMind-2-Pro
We did not release a Chat version yet because it regressed. Release Later.
Follow us to know when BananaMind 2 Ultra releases and support us at
@Banaxi-Tech
@vovaRL
@DedeProGames
- 19 replies
replied to their post 15 days ago
Moreover, the implications are that finetuning models already pretrained on structure could outperform ones starting from scratch.
posted an update 15 days ago
Post
167
If you lack ideas for a cool model, here's one.
Train a model from scratch on wikipedia with one twist: the tokenizer changes the actual token ids used on every sample fed. If somehow still learns English, you have made an astonishing discovery.
You would have answered the question: Can a model learn human languages from structure alone?
Train a model from scratch on wikipedia with one twist: the tokenizer changes the actual token ids used on every sample fed. If somehow still learns English, you have made an astonishing discovery.
You would have answered the question: Can a model learn human languages from structure alone?
- 2 replies
replied to Banaxi-Tech's post 16 days ago
I'm pretty sure you need to check the memory window of that model before training. Check falcon-h1-tiny blogpost to understand what I mean.
reacted to Banaxi-Tech's post with ๐ฅ 16 days ago
Post
2749
Today we wanted to release BananaMind 2 Pico, our smallest model yet at ~0.9M parameters. Instead, we accidentally ran a very expensive experiment on what happens when you push a tiny model way past its useful token budget.
Short version: we trained on 200B tokens (~222K:1 tokens-per-parameter). The model peaked at 20B tokens with an INT Index of 4.55, then degraded monotonically over the next 160B to 3.31 โ a 27% regression. Three of four Open SLM benchmarks were worse at the end of training than they were at 10% through.
The useful compute-optimal range for Pico-tier models looks like ~22Kโ30K tokens per parameter. Ratios like 7K:1, 15K:1, and 22K:1 all work fine โ TinyStories and most sub-3M community models sit in this range. Push much further and benchmarks start rotting.
Follow us for more:
BananaMind
@vovaRL
@Banaxi-Tech
Full writeup with all checkpoints, the Chinchilla-ratio control run, and the schedule-vs-overtraining analysis: https://huggingface.co/blog/Banaxi-Tech/ovdadadadd
And if anyone, i dont know the reason why you would, wants the 20B token checkpoint reply and ill upload it as BananaMind 2.1 Pico EXP
Short version: we trained on 200B tokens (~222K:1 tokens-per-parameter). The model peaked at 20B tokens with an INT Index of 4.55, then degraded monotonically over the next 160B to 3.31 โ a 27% regression. Three of four Open SLM benchmarks were worse at the end of training than they were at 10% through.
The useful compute-optimal range for Pico-tier models looks like ~22Kโ30K tokens per parameter. Ratios like 7K:1, 15K:1, and 22K:1 all work fine โ TinyStories and most sub-3M community models sit in this range. Push much further and benchmarks start rotting.
Follow us for more:
@vovaRL
@Banaxi-Tech
Full writeup with all checkpoints, the Chinchilla-ratio control run, and the schedule-vs-overtraining analysis: https://huggingface.co/blog/Banaxi-Tech/ovdadadadd
And if anyone, i dont know the reason why you would, wants the 20B token checkpoint reply and ill upload it as BananaMind 2.1 Pico EXP
- 3 replies
posted an update 17 days ago
Post
1960
Random corporate secret of tonight:
Try overfitting a tiny model on billions of high-quality datapoints: you can't. You can do 100 epochs and see the model still improving.
You're welcome.
Try overfitting a tiny model on billions of high-quality datapoints: you can't. You can do 100 epochs and see the model still improving.
You're welcome.
- 1 reply
posted an update 18 days ago
Post
859
- GLM 5.2
- Flux 3
- New Qwen model
- New small model leaderboards
- Lots of people finetuning smol models.
- Some even under 12 year olds clauders are here (was not on my bingo card this year)
- ChatGPT's Sol became a lot faster this week
- LFM2.5 2.6b
- Kimi K3 (though only a few will run it)
- New Ling 3.0 Tiny
- New video model that is making south park videos?
- Deepseek v4 flash being more honest than bigger models
- The new model from meta
Everything Everywhere All At Once
- Flux 3
- New Qwen model
- New small model leaderboards
- Lots of people finetuning smol models.
- Some even under 12 year olds clauders are here (was not on my bingo card this year)
- ChatGPT's Sol became a lot faster this week
- LFM2.5 2.6b
- Kimi K3 (though only a few will run it)
- New Ling 3.0 Tiny
- New video model that is making south park videos?
- Deepseek v4 flash being more honest than bigger models
- The new model from meta
Everything Everywhere All At Once
- 1 reply