Article
Aether-7B-5Attn: A 100% Open-Source Sovereign Foundation Model — and a Controlled Experiment in Heterogeneous Attention
FINAL-Bench
• • 21
Dave White
darkvplayer
·
AI & ML interests
Flux, Stable Diffusion, T2I, T2V, T23D, T2M, anything that basically is magic from text.
Trained some loras, learned some ML concepts. Using unsloth studio for simplicity.
Recent Activity
liked a model about 1 month ago
numz/SeedVR2_comfyUI liked a model about 1 month ago
zai-org/GLM-5.2Organizations
None yet
upvoted an article 2 days ago
liked 2
models about 1 month ago
reacted to philipp-zettl's post with 👍 3 months ago
Post
2694
I've been cooking something neat over the past weeks 👨🍳
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
Final models get exported in
- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
We all know that training LLMs requires a lot of resources and especially a lot of compute in form of GPUs, or is super slow and inefficient when done on CPUs.
The big players use giant clusters of Nvidia H100s.
But if I look at the profiles of my fellow home brewers, all we can get our hands on are those pesky consumer RTX's. If you're lucky you got yourself a 5080 with 16GB VRAM or something.
To be frank, I don't have that 1.3k disposable cash laying around ¯\_(ツ)_/¯
But I can write rust and like building ML libraries.
So I asked myself the question(s):
- can I train SMLs at home on my hardware?
- How hard can it be to build a ML library that can stream data between RAM and VRAM on demand, like llama.cpp's unified memory feature [^1]?
- how hard can it be to implement bf16 support?
The answers are wild, trust me!
Image 1: Metrics form last nights build on my "tiny" RTX 2060 (6 GB VRAM)
Image 2: Metrics from my most recent build on my RTX 4070 Laptop (8GB VRAM)
The majority of my time went into the shared memory, but it's stable and I'm very excited!
Here some debug logs, a la "trust me bro"
----
Currently available: 1112735744, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6744 MB / 7805 MB
Data on GPU: 1641 MB
Grads on GPU: 3459 MB
CPU Offloaded: 18230 MB
---------------------------------
Currently available: 1079181312, attempting to reclaim: 1073741824
--- VRAM STATE [backward pass] ---
Driver Used: 6776 MB / 7805 MB
Data on GPU: 1561 MB
Grads on GPU: 3279 MB
CPU Offloaded: 18590 MB
-----------------------------Final models get exported in
safetensors format and are compatible with PyTorch and transformers, for accessibility.- [^1]: https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#unified-memory
- 1 reply
liked a
Space 10 months ago
liked a
model 12 months ago
liked a
model about 1 year ago
liked 2
Spaces about 1 year ago
liked 3
models about 1 year ago
liked a
Space about 1 year ago
liked a
model about 1 year ago
liked a
model over 1 year ago
liked 5
Spaces over 1 year ago
EXPERIMENTAL
🚀
26
Generate captions for images using text prompts
TripoSG
🔮
883
Create a textured 3D model from a single image
MV Adapter I2MV SDXL
🌍
228
Generate multi-view images from a single image
Diffusion Self Distillation
🦀
191
Tuning-free subject-driven generation
Joy Caption Pre Alpha
💬
1.46k
Generate descriptive captions for any image