Open-source AI is free, but most people still can’t use it

Date:


Open-source AI sounds like an open door. Developers can grab code for models like Meta’s Llama, Stability AI’s Stable Diffusion, or anything from Mistral, and start building. The tools are free to download and use. The idea is to give everyone – from solo programmers to small teams – the same tools big companies have. But things aren’t that simple.

Hardware isn’t free

As detailed in an article on Substack, Cambridge University’s Dr. Saffron Huang uses Stable Diffusion for her computer vision research with a custom rig that comprises of eight NVIDIA RTX 4090 GPUs, at a cost of over £20,000. “There’s a profound irony here,” she says. “The code is freely available, yet the computational resources needed to train, fine-tune, or sometimes even run these models create a new technological aristocracy.”

The gap becomes wider when training large models. Meta’s Llama 3, with 70 billion parameters, needed thousands of GPUs running non-stop for months in its training phase. That power bill alone would ruin a small business. Surely independent researchers don’t stand a chance?

“We’re witnessing a bifurcation of the AI community,” says Dr. Jakob Uszkoreit, co-founder of Inceptive. “On one side, you have organisations with access to vast computational resources who can advance the frontier; on the other, you have everyone else who must adapt pre-trained models in severe computational constraints.”

Environmental costs add to the problem. One large model can produce as much carbon as five cars over their lifetime. The need for compute creates a new kind of gate-keeping that hits poorer and climate-vulnerable regions hardest.

It’s not equal around the world

The hardware problem is most apparent when considered on geographical terms. In the Global South, access to powerful GPUs and fast, stable internet is limited. That gap has led Dr. Timnit Gebru to call the trend “algorithmic colonialism.”

In Nigeria, software engineer Chioma Onyekwere faces regular power outages while trying to build a diagnostic tool with open-source AI. “The irony isn’t lost on me,” she says. “The technologies could theoretically benefit under-served communities most, yet we face insurmountable barriers to implementation.”

Only 40% of people in Africa have reliable internet, based on ITU data. And even fewer can download and run large AI models. Cryptocurrency mining and supply chain problems have made GPUs even harder to get.

Cloud providers like AWS and Google Cloud have added more data centres, but many regions experience latency issues that make real-time AI impossible. “It’s a kind of digital redlining,” says Dr. Rumman Chowdhury. “Open-source AI has unintentionally created a two-tier system of access that reinforces existing global inequities.”

Skills are a wall too

Even with internet and hardware, there is still a significant requirement for specialised technical knowledge. Many open models require the understanding of machine learning and programming. Hugging Face and other tools can help, but the barrier to entry remains high. “The cognitive load of working with these models is immense,” says Dr. Charles Sutton from the University of Edinburgh. “Even with streamlined interfaces, you’re still dealing with complex hyperparameter optimisation, training dynamics, and model architecture decisions that require years of specialised education.”

Most contributors to open-source AI projects have advanced degrees, according to GitHub. “We must acknowledge that ‘open’ doesn’t automatically mean ‘accessible,’” says Dr. Juliana Peña of the Mozilla Foundation. “When access requires advanced mathematical knowledge or programming skills, we’re still maintaining exclusivity – just through different means.”

There are free courses online, but most expect users to already know how to code, and of course, accessing them requires a steady internet connection.

A shared approach to hardware

To fix the hardware problem, some groups have created shared compute setups like the one in Berlin, where the EleutherCollective runs a GPU cluster that artists and researchers can use. “We’re attempting to re-imagine the relationship between communities and computing power,” says Frieda Schmidt, one of the founders. “Rather than individuals struggling to purchase expensive hardware, we pool resources and democratically govern their allocation.”

Other examples include SuperComputing Commons in Barcelona and the Computational Democracy Project in Seoul, setups which focus on projects that have a positive social impact.

“Compute cooperatives represent a middle path between market-driven exclusivity and purely state-controlled infrastructure,” says economist Dr. Kate Raworth. “They embody the commons-based approach that could reorient technology toward genuine public benefit.”

Such projects are known to struggle with upkeep, rising electricity bills, and old hardware. Without regular funding and technical support, their future remains shaky.

Running AI without owning it

Another route to easier access to AI is open inference. Companies like Together.ai and Hugging Face let developers run models remotely through APIs. “We’re separating model ownership from model utility,” says Hugging Face CEO Clément Delangue. “Anyone with an internet connection can now use state-of-the-art AI through simple API calls, irrespective of their hardware constraints.”

Hugging Face has helped developers in low-resource areas launch translation tools in Mongolia and farming support apps in India. But such help comes with limits, with APIs often having use caps, and the problem of poor internet connectivity still acts as a block to access. Plus, there’s the issue of reliance on outside services creating dependency.

“We must ask whether API access truly constitutes democratisation,” says Dr. Nathan Schneider from the University of Colourado Boulder. “Are we creating more equitable access or merely shifting from hardware dependency to service dependency?”

Governments get involved

Some governments have tried to help, with the EU putting €2.5 billion into compute infrastructure for open-source AI, and Canada’s national AI plan funding public compute at large institutions. But small groups often don’t qualify, applications require paperwork and credentials that many developers don’t have. “Public computing infrastructure should be treated as essential as public libraries,” says Dr. Yoshua Bengio. “We need a fundamental shift in how we conceptualise access to computational resources in the 21st century.”

Elsewhere several countries have tried different schemes, such as Finland guaranteeing compute time to projects tied to the UN’s development goals, and Uruguay adding AI tools to its schools’ curriculum. But both plans focus less on who the users are and more on what they are trying to achieve.

Companies still call the shots

When companies release open-source models they are taking positive steps towards AI democratisation, but it doesn’t mean models are necessarily easy to use or access. Meta’s Llama 2 public release came with fewer restrictions than earlier versions, but users still needed expensive hardware to run the model effectively. It also came with a restrictive licence for commercial usage.

“There’s a spectrum of openness that’s often collapsed in public discourse,” says Dr. James Grimmelmann. “The difference between Meta’s Llama licence and something like Apache 2.0 or GPL is substantial, with implications for who can meaningfully build on these technologies.”

“There’s a cynical reading of corporate open-source AI,” says Dr. Meredith Whittaker. “By releasing models without ensuring broad access to the necessary computing resources, companies enjoy the reputational benefits of openness while maintaining in fact exclusivity through licence restrictions and compute barriers.”

Some companies offer help such as NVIDIA offering GPU grants, Google’s offer of free cloud credits, and Stability AI’s support of computing co-operatives in Southeast Asia and Africa. “We recognise that releasing open models is only half the equation,” says Emad Mostaque, CEO of Stability AI. “Without addressing the compute gap, open-source remains an empty promise for much of the world.”

Smaller models, smarter design

Some researchers focus on building smaller, more efficient models. Dr. Laura Montoya at UCL (University College London) calls this ‘frugal AI.’ “We’re challenging the assumption that bigger is always better,” she says. “Through careful architecture design and knowledge distillation, we can create models that run on a fraction of the compute while maintaining most capabilities.”

Tools like quantisation, pruning, and distillation help shrink models but keep performance levels high: Mistral 7B and Microsoft’s Phi-2 run well on lower-end systems, for example.

“The future of accessible AI doesn’t just lie in distributing compute more equitably, but in fundamentally rethinking our approach to model design,” says Dr. Yann LeCun. “We need to draw inspiration from human cognition, which achieves remarkable capabilities with relatively modest energy requirements.”

Data still a bottleneck

Models might be open and easily available, but access to the necessary training data often isn’t. “We’ve created a situation where models are open but the data needed to make them useful often isn’t,” says Dr. Margaret Mitchell. “This creates another layer of exclusion that disproportionately affects marginalised communities.”

Most training data is in English or reflects Western viewpoints. High-quality data costs money and time to collect, with many organisations struggling to afford it.

Some sources of training data are available under open-source licences, notably from organisations that are independent of the big AI companies. Mozilla’s Common Voice and the Masakhane projects have helped by crowdsourcing speech data and language resources. “Data commons represent a crucial complement to open-source models,” says Dr. Rada Mihalcea. “By creating collectively governed repositories of diverse, ethically sourced data, we can begin to address the input side of the accessibility equation.”

Making ‘open’ actually open

Fixing the gap in open-source AI will take effort on many fronts. Models need to be more efficient, policies need to treat compute like public infrastructure, shared GPU pools need stable support, and companies should go beyond model releases and help build access.

“The democratisation of AI isn’t a technical challenge so much as a socio-political one,” says Dr. Kate Crawford. “It requires us to re-imagine our relationship with technology, challenging assumptions about ownership, access, and governance.”

Education matters too. More people need to learn how to use these tools and that means building learning pathways outside traditional schools.

“We’re at an inflection point in the development of AI,” says Dr. Deb Raji. “The decisions we make now about accessibility and infrastructure will determine whether open-source AI fulfils its democratising potential or merely reproduces existing power dynamics in new forms.”

Open-source AI won’t be truly open until more people can actually use it. With the right mix of technical change, community support, and public policy, that future is still in reach.

(Photo by Shahadat rahman)

See also: Why is 40-year-old programming language Ada hot again?

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is co-located with other leading events including Intelligent Automation Conference, BlockX, Digital Transformation Week, and Cyber Security & Cloud Expo.

Explore other upcoming enterprise technology events and webinars powered by TechForge here.

Tags: ai, coding, developer, featured, open source, software development

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Share post:

Subscribe

spot_imgspot_img

Popular

More like this
Related

Former USA TODAY editor David Mazzarella, who helped give paper its hard news edge, dies

Former USA TODAY editor David Mazzarella, who helped...

Jeremiah Owusu-Koramoah Given Worrisome Future Prediction

  The Cleveland Browns had a rising star in...

Comparing Apple M4‑powered Macs (July 2025)

MacBook Air 13″ (2025)10‑core CPU (4 P + 6 E), 8‑ or 10‑core...