In brief
- Social media platform X has open-sourced its Grok-based transformer model, which ranks For You feed posts by predicting user actions without relying on hand-engineered features.
- The GitHub repository includes all code determining organic and advertising post visibility, built in Rust and Python for modular retrieval and scoring.
- The release follows owner Elon Musk’s pledge to update the algorithm every four weeks, accompanied by comprehensive developer notes.
Elon Musk’s social media platform X delivered on its promise to pull back the curtain on one of social media’s most closely guarded secrets Tuesday, releasing the machine learning architecture that determines what posts appear in users’ feeds.
“We have open-sourced our new 𝕏 algorithm, powered by the same transformer architecture as xAI’s Grok model,” X’s engineering team tweeted.
“We know the algorithm is dumb and needs massive improvements, but at least you can see us struggle to make it better in real-time and with transparency,” Musk tweeted following the release. “No other social media companies do this.”
The release follows through on a pledge Musk made last week when he posted that he would “make the new X algorithm, including all code used to determine what organic and advertising posts are recommended to users, open source in 7 days.”
He promised updates would be “repeated every 4 weeks, with comprehensive developer notes, to help you understand what changed.”
The GitHub repository details a Grok-based transformer model that ranks X’s ‘For You’ feed posts by predicting user actions like likes and replies, using end-to-end machine learning without hand-engineered features, built in Rust and Python for modular retrieval and scoring.
The algorithm retrieves content from two sources: in-network posts from accounts users follow and out-of-network posts discovered through ML-based retrieval, combining both through a scoring system that predicts engagement probabilities for each post.
Midhun Krishna M, co-founder and CEO of LLM cost tracker TknOps.io, said the open-source release could change industry standards.
“By exposing the Grok-based transformer architecture, X is essentially handing developers a blueprint to understand, and potentially improve upon, recommendation systems that have been black boxes for years,” he told Decrypt. “This level of transparency could force other platforms to follow suit or explain why they won’t.”
“Creators can learn what works and adjust without blindly gaming the system, while clearer incentives benefit regular users and lead to better content,” he added.
When asked whether the open-source code could help users determine what makes posts go viral, Grok itself analyzed the algorithm and identified five key factors.
These include engagement predictions based on user history for likes and reposts, content novelty and relevance with timely personalized posts scoring higher, diversity scoring that limits repeated authors, a balance between followed accounts and ML-suggested posts, and negative signals from blocks and mutes that lower scores.
X under scrutiny
The release comes amid heightened scrutiny of X’s AI initiatives, as last week, X revoked API access for InfoFi projects that rewarded users for platform engagement, with Head of Product Nikita Bier declaring the company would “no longer allow apps that reward users for posting on X” due to AI-generated spam concerns.
Recently, X restricted Grok’s image generation and editing features to paid subscribers only and implemented technical measures to prevent editing images of real people after the chatbot was used to create non-consensual sexualized images, including those of minors, prompting regulators around the world to open investigations that could lead to enforcement action.
Daily Debrief Newsletter
Start every day with the top news stories right now, plus original features, a podcast, videos and more.
Source: https://decrypt.co/355108/elon-musks-x-open-sources-grok-powered-algorithm-driving-for-you-feed
