ggml.ai joins Hugging Face to ensure the long-term progress of Local AI

[ggerganov]

Announcement

We are happy to announce that ggml.ai (the founding team of llama.cpp) are joining Hugging Face in order to keep future AI truly open.

Georgi and team are joining HF with the goal of scaling and supporting the ggml/llama.cpp community as Local AI continues to make exponential progress in the coming years.

Summary / Key-points

• The ggml-org projects remain open and community driven as always
• The ggml team continues to lead, maintain and support full-time the ggml and llama.cpp libraries and related open-source projects
• The new partnership ensures long-term sustainability of the projects and will help foster new opportunities for users and contributors
• Additional focus will be dedicated on improving user experience and integration with the Hugging Face transformers library for improved model support

Why this change?

Since its foundation in 2023, the core mission of ggml.ai has continuously been to support the development and the adoption of the ggml machine learning library. Over the past 3 years, the small team behind the company has been doing its best to grow the open-source developer community and to help establish ggml as the definitive standard for efficient local AI inference. This was achieved through strong collaboration with individual contributors, as well as with partnerships with model providers and independent hardware vendors. As a result, today llama.cpp has become the fundamental building block in countless projects and products, enabling private and easily-accessible AI on consumer hardware.

Throughout this development, Hugging Face stood out as the strongest and most supportive partner of this initiative. During the course of the last couple of years, HF engineers (notably @ngxson and @allozaur) have:

• Contributed several core functionalities to ggml and llama.cpp
• Built a solid inference server with polished user interface
• Introduced multi-modal support to llama.cpp
• Integrated llama.cpp into the Hugging Face Inference Endpoints
• Improved compatibility of the GGUF file format with the Hugging Face platform
• Implemented multiple model architectures into llama.cpp
• Helped ggml projects with general maintenance, PR reviews and more

The teamwork between our teams has always been smooth and efficient. Both sides, as well as the community, have benefited from these joint efforts. It only makes sense to formalize this collaboration and make it stronger in the future.

What will change for ggml/llama.cpp, the open source project and the community?

Not much – Georgi and team will continue to dedicate 100% of their time maintaining ggml/llama.cpp. The community will continue to operate fully autonomously and make technical and architectural decisions as usual. Hugging Face is providing the project with long-term sustainable resources, improving the chances of the project to grow and thrive. The project will continue to be 100% open-source and community driven as it is now. Expect your favorite quants to be supported even faster once a model is released.

Technical focus

Going forward, our joint efforts will be geared towards the following objectives:

• Towards seamless “single-click” integration with the transformers library
  The transformers framework has established itself as the ‘source of truth’ for AI model definitions. Improving the compatibility between the transformers and the ggml ecosystems is essential for wider model support and quality control.

• Better packaging and user experience of ggml-based software
  As we enter the phase in which local inference becomes a meaningful and competitive alternative to cloud inference, it is crucial to improve and simplify the way in which casual users deploy and access local models. We will work towards making llama.cpp ubiquitous and readily available everywhere, and continue partnering with great downstream projects.

Long term vision

Our shared goal is to provide the building blocks to make open-source superintelligence accessible to the world over the coming years. We will achieve this together with the growing Local AI community, as we continue to build the ultimate inference stack that runs as efficiently as possible on our devices.

[julien-c]

welcome @ggerganov and team!

we're happy to get the chance to continue supporting the awesome llama.cpp community 🔥

[rabbidave]

Appreciate y'all immensely; bringing hope and long-term vision for local-first technology to plebeians 👑

[giladgd]

Wanted to use this opportunity to say that you guys rock!
@ggerganov @ngxson @allozaur @pwilkin @CISC @0cc4m @JohannesGaessler @jeffbolznv @am17an @max-krasnyansky @slaren @ericcurtin Thank you for all the great work you do pushing llama.cpp forward! ❤️

I don't usually mass tag like this (sorry if I forgot to tag someone), just wanted to show my appreciation for your hard work and dedication on this project that I personally noticed every other day for the past few years. Thank you!

[ericcurtin]

Thank you @giladgd although I'm not going to take credit for this :)

These days my goal is to ensure Docker Model Runner never forks llama.cpp or related ggml projects (there was a minor fork which I pushed hard to abolish and succeeded, the final PR went in not that long ago docker/model-runner#541) and upstreams first when necessary, the golang layer is important for distributing models via OCI registries.

[Solido]

sacré mov @julien-c !

[max-krasnyansky]

Awesome news! Super excited to be part of this! Lot more cool stuff is coming!

[allozaur]

LEEET'S GOOOOOOOO!!!

It's been such an honour and privilege to work on llama.cpp and this is the best news for the truly open AI ecosystem and democratising local AI.

[pwilkin]

That's really cool! Great news for @ggerganov and the team!

[hoverflow]

congrats!

[EAddario]

Congratulations to the entire ggml.ai team and what a fantastic move by HF!

[ddh0]

Wahoo!

[Mihaiii]

This is awesome! Congrats! 🇪🇺🇪🇺🇪🇺

[steampunque]

Great to hear this news, congrats all around to the entire team and thanks to everyone for all your dedicated efforts over the years!

[JohannesGaessler]

Good to hear that you managed to secure a partnership for the sustainability of the project! Though I am not part of the formal ggml organization I will happily continue to cooperate on our shared goals.

[ggerganov]

Thank you, Johannes! I am grateful for your major contributions to the project and I enjoy our closer collaboration lately.

[davanstrien]

Very excited about this!

[ali0une]

Congrats. All the best for this wonderful project. Keep up the good work!
Love you guys ❤️

[giladgd]

This is amazing news!
HuggingFace seems like a natural fit for the projects and the community

[danielhanchen]

Congrats Georgi, GGML team and Hugging Face! This is amazing news.

Llama.cpp is what made open-source local AI what it is today. Excited to see what's coming 🎉🦙

[ericcurtin]

Congrats to all at GGML and Hugging Face. Excited to see all the GGML efforts move from strength to strength. Glad to see Hugging Face as the acquiring company, I trust the community will remain strong.

[ericcurtin]

I like the long term vision @ggerganov makes in the announcement here. Future directions make a lot of sense.

[davidmezzetti]

Congratulations on this news! All the best.

It will be great to have strong Python support for llama.cpp. The current options are limited to either hosting via API (llama.cpp, vllm, ollama etc) or using the largely unmaintained llama-cpp-python library.

[mcraveiro]

Amazing news, congrats guys, more than deserved. Thanks for all your hard work so far. One minor question though, I was a bit confused with this though to be fair I do not know too much about llama.cpp's internals:

Towards seamless “single-click” integration with the transformers library
The transformers framework has established itself as the ‘source of truth’ for AI model definitions. Improving the compatibility between the transformers and the ggml ecosystems is essential for wider model support and quality control.

Isn't this a python library? I thought llama.cpp aimed at remaining lean and mean c++ only.

Thanks for your time

[Mainframework]

Congratulations, you deserve it. Still can´t thank you enough for the great work and contributions.
P.S: Huggingface just got lucky :)

[ariym]

Congratulations! Been a GGML believer since whisper.cpp in 2022!

[stevennyman]

Congratulations! Will whisper.cpp continue to be supported? I don't see any mentions of it in the announcement.

[gardner]

Congratulations! 🎉

[fenn]

was this even discussed publicly before it happened? we should consider the pros and cons for the project from a variety of perspectives.
i don't understand the supposed need for secrecy in an open source project being acqui-hired by a corporation. this feels bad and unilateral.
the maintaining organization being acquired by a US corporation means that now the llama.cpp project is effectively under US jurisdiction, whereas before it was not. this has huge implications and should have been discussed openly.

90% worry and concern among users in response to this announcement: https://old.reddit.com/r/LocalLLaMA/comments/1r9vywq/ggmlai_has_got_acquired_by_huggingface/

this announcement comes on the heels of leaks showing "Open"AI directly collaborating with the US government to secretly flag and report "suspicious" users to government agencies based on their political exposure: https://vmfunc.re/blog/persona

who's to say that HF isn't already implementing this sort of extralegal of identity and political check? it will happen in the future. let's not deliberately set ourselves up to get burned again.

to everyone else: please, be aware that your independence has value.

[paveljee]

While I get it's a strong move for the GGML team and don't feel like I'm in a position to comment because they own the thing anyway, albeit with OSS community contributions; I cannot help but share the sad sentiment.

[invisiblebydaylight]

Is there any change in ownership of the code repository? Or is it still considered 'owned' by the same 'ggml-org'? At the very least, clarification of who exactly owns the code now should probably be in the README.

[vpk11]

Congratulations 🎉

[TwoTwenty]

hopefully it does not go the way of github and Microsoft acquisition

[nnikolov3]

As a Bulgarian, anywhere I go I talk about your work! I appreciate you! You are the SW "Stoichkov" for me! :)

[ericcurtin]

Love the analogy! :)

[bobqianic]

Congratulation!

[beginor]

Congratulations!

[sandkumaCode]

Congratulations! would love to have cpp and rust api with a bit more documentation

[NeoZhangJianyu]

Congratulations! Hope see new change of llama.cpp after the joining. :)

[andyceo]

Congrats! Hope this move all the AI world to the bright good future!