Offline AI

[LeanBitLab]

Model Name

Visheratin T5 Tiny

Device Specs (RAM/CPU)

4GB Ram sd720g Cpu

Feedback / Issue

Welcome to Offline AI discussion!

[ALMIGHTY-VC]

I tried the Xenova grammar small quantized based on T5, Pszemraj's grammar small quantized and yeah both don't work as intended.So they are out of consideration for now I think.

[LeanBitLab]

Only Visheratin T5 Tiny worked for me too

[AnaelR]

Only Visheratin T5 Tiny worked for me too

I tried it today and it did not work for me

[LeanBitLab]

Only Visheratin T5 Tiny worked for me too

I tried it today and it did not work for me

you can expect gguf model support in next release

[ALMIGHTY-VC]

In this month my exam AR going on so,consider me mostly offline.Still,here are some of my interesting finds:- 1)Prithivida gramformer(quantized if available I'm unable to find it in this haste) 2)Falcon H1 Tiny 90M instruct 3)HRM Grammar Light V2(2026 edition) Also another question have you tried the qwen offline models? How about implementing a single model.onnx it will be better for the memory and processes.

…

On Sat, 31 Jan, 2026, 11:38 pm LeanBitLab, ***@***.***> wrote: Only Visheratin T5 Tiny worked for me too — Reply to this email directly, view it on GitHub <#16 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2YYNFFD6G4QCBEGKJNPX334JTVRZAVCNFSM6AAAAACTQYXALCVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKNRVHE4DEMY> . You are receiving this because you commented.Message ID: ***@***.***>

[LeanBitLab]

I find only Qwen gguf models, when I tried the gguf engine with the Qwen model, only models of 600mb or above were barely usable since the gguf engine wasn't that customisable, I didn't find any ONNX models for Qwen

[vishaldebnath19-commits]

Instead of ONNX
Llama 1b . gguf
Works faster then other.
In offline ai
Please removed ONNX features instead please add gguf
There are so many options in gguf.
All downloadable

[LeanBitLab]

Instead of ONNX
Llama 1b . gguf
Works faster then other.
In offline ai
Please removed ONNX features instead please add gguf
There are so many options in gguf.
All downloadable

Actually, GGUf models were the first engine I used. I tried multiple GGUf models too. The issue is that the GGUf engine isn't tweakable enough for proofreading. There weren't any lightweight, proofreading-optimized GGUf models that were better than ONNX models.

[vishaldebnath19-commits]

First of All Hats off and Many Many Many Thank to You & LeanBitLab & Developer's for Creating such a Handsome Customizable keyboard.

So
As you tried using Qwen gguf .I Have a Good Smartphone ,And I have tried many gguf in my phone ,As per my experience Qwen gguf is the most slowest gguf. There are tons of gguf like EmbeddingGemma 300M or llama many more. Ggguf 1b models may work solw in many smartphone but 300m or 110m Gguf will work faster.

Instead you can add
task / litertlmfile. Which used in Google offline ai Edge Gallery
Options like TONE change / proofreading is important at offline ai keyboard.For this Custom AI Keys
Is Important please add this feature .

And can you please mention me on which versions you have used gguf feature so that i can try or download

[LeanBitLab]

First of All Hats off and Many Many Many Thank to You & LeanBitLab & Developer's for Creating such a Handsome Customizable keyboard.

So
As you tried using Qwen gguf .I Have a Good Smartphone ,And I have tried many gguf in my phone ,As per my experience Qwen gguf is the most slowest gguf. There are tons of gguf like EmbeddingGemma 300M or llama many more. Ggguf 1b models may work solw in many smartphone but 300m or 110m Gguf will work faster.

Instead you can add
task / litertlmfile. Which used in Google offline ai Edge Gallery
Options like TONE change / proofreading is important at offline ai keyboard.For this Custom AI Keys
Is Important please add this feature .

And can you please mention me on which versions you have used gguf feature so that i can try or download

I tried other models like Smollm, Gemma, and Tinylamma. I even tried to train a specified proofreading model, since the GGUf engine wasn't tweakable enough for proofreading or any specific task. The custom prompt wasn't enough, even with 1.5b qwen given mediocre proofreading results. If there exists a good, lightweight text correction specific model (less than 500 MB), I would gladly reuse the GGUf engine. I don't want the keyboard to consume too much battery or processing power.

[vishaldebnath19-commits]

This is how my Heliboard Learn Type keyboard look like absolutely stunning.
I have tired onnx may be you find it better. But it has some flaws , on onnx there is no option like rewrite tone,Tone chang e and many more.
I have noticed that for
offline ai Google keyboard use
Gemma.
You can also integrate " Gemma3-1B-IT " which comes under 500 mb with our keyboard and the best part Gemma consume very less battery and mobile space. Gemma3-1B-IT runs on "task" or ".litertlm" formet.
In know gguf formet is slow ,But task" or ".litertlm works better in every smartphone.
Gemma3-1B-IT has features like Free form ,Rewrite tone,Summarize text

Tone: Casual

Formal

Casual

Friendly

Polite

Enthusiastic

Concise

[vishaldebnath19-commits]

Thanks To The Best Developer Who Brought Ai In Keyboard,which is totally offline.
Hats off To You.
I have checked several gguf model and found out that
LFM2-350M-Q5_K_M.gguf is the best for offline ai.
Comes with small size.
Easy to use,fast replay

[AnaelR]

I’m using Qwen2.5-1.5B Q4_K_M and it’s okay (in French it’s not very efficient, but in English it’s quite good), but I don’t know if it’s the model or the base prompt. When I write in French and click the AI button, it translates the text into English. I added a system instruction to explicitly keep the original language. Also, there is no English as a target language for translation in the settings.

[LeanBitLab]

I ll add an custom target option, i own an 4gb ram device so you guess the how far i can test the offline models, my plan is to add some experimental offline model improving stuffs and wait for users to test it open the issues😁