The design can run a big neural network more efficiently than banks of GPUs wired together. But manufacturing and running the chip is a challenge, requiring new methods for etching silicon features, a design that includes redundancies to account for manufacturing flaws, and a novel water system to keep the giant chip chilled.
To build a cluster of WSE-2 chips capable of running AI models of record size, Cerebras had to solve another engineering challenge: how to get data in and out of the chip efficiently. Regular chips have their own memory on board, but Cerebras developed an off-chip memory box called MemoryX. The company also created software that allows a neural network to be partially stored in that off-chip memory, with only the computations shuttled over to the silicon chip. And it built a hardware and software system called SwarmX that wires everything together.
Photograph: Cerebras
“They can improve the scalability of training to huge dimensions, beyond what anybody is doing today,” says Mike Demler, a senior analyst with the Linley Group and a senior editor of The Microprocessor Report.
Demler says it isn’t yet clear how much of a market there will be for the cluster, especially since some potential customers are already designing their own, more specialized chips in-house. He adds that the real performance of the chip, in terms of speed, efficiency, and cost, are as yet unclear. Cerebras hasn’t published any benchmark results so far.
“There’s a lot of impressive engineering in the new MemoryX and SwarmX technology,” Demler says. “But just like the processor, this is highly specialized stuff; it only makes sense for training the very largest models.”
Cerebras’ chips have so far been adopted by labs that need supercomputing power. Early customers include Argonne National Labs, Lawrence Livermore National Lab, pharma companies including GlaxoSmithKline and AstraZeneca, and what Feldman describes as “military intelligence” organizations.
This shows that the Cerebras chip can be used for more than just powering neural networks; the computations these labs run involve similarly massive parallel mathematical operations. “And they’re always thirsty for more compute power,” says Demler, who adds that the chip could conceivably become important for the future of supercomputing.
David Kanter, an analyst with Real World Technologies and executive director of MLCommons, an organization that measures the performance of different AI algorithms and hardware, says he sees a future market for much bigger AI models. “I generally tend to believe in data-centric ML [machine learning], so we want larger data sets that enable building larger models with more parameters,” Kanter says.
When federal judge James Boasberg dismissed the Federal Trade Commission’s antitrust lawsuit against Facebook in June, he gave the agency pretty specific instructions on how to salvage it. The problem, he wrote in his opinion, was that the FTC hadn’t offered even the barest evidence that Facebook is a monopoly, beyond the vague claim that it “maintained a dominant share of the US personal social networking market (in excess of 60 percent).” As Boasberg noted, that inexplicably left some basic questions unanswered, such as: 60 percent of what? Who makes up the leftover 40 percent? It was a bit like accusing a driver of speeding without even mentioning the speed limit.
To get back into court and advance to the next stage of litigation, the FTC would have to come back with something a lot more specific. That presented an interesting early assignment for Lina Khan, who was confirmed as commissioner of the agency a mere two weeks before Boasberg issued his ruling. (Facebook has sought to have Khan recused from the case on the basis of her public criticism of big tech companies before her current job, though experts see little chance of that succeeding.)
On Thursday, the FTC filed its revised complaint answering those previously unanswered questions. While it’s impossible to predict how a given judge will rule, the new material seems likely to satisfy Boasberg and keep the case alive. “To my eye, they’ve scratched Boasberg’s itch,” said Paul Swanson, an antitrust attorney in Denver. Facebook, he said, may not be able to avoid “a long slog of document productions and depositions.”
To prove that Facebook is a monopoly for legal purposes, the FTC doesn’t have to show that it’s literally the only social network. They have to show that it has “market power.” In a nutshell, having market power means you face so little competition that you can do things your customers don’t like without losing any business. It’s one of the main reasons antitrust law exists: When there isn’t enough competition, companies will stop trying to please their customers and start trying to squeeze them. Think about how frustrating it is when your internet provider raises prices and you realize no one else serves your neighborhood. That’s market power.
There are two ways to show market power: indirect evidence and direct evidence. Indirect evidence usually refers to dominant market share. (That might sound counterintuitive, but the reason it’s indirect is because being big on its own doesn’t prove a company is doing anything wrong—it just raises the strong possibility.) In its initial complaint, the FTC only offered indirect evidence, and very little of it: that feeble 60 percent statistic, which Boasberg ruled was inadequate. The revised complaint, on the other hand, goes into great detail on market share. Drawing on data from the analytics company Comscore—which, the complaint notes, Facebook itself relies on—the FTC argues that just about any way you slice it, Facebook controls a dominant chunk of the market for “personal social networking services.” According to the Comscore data, Facebook has accounted for more than 80 percent of time spent since 2011, at least 70 percent of daily active users, and at least 65 percent of monthly active users.
The new complaint also tightens up the FTC’s definition of the market itself, which is another crucial part of any monopolization case. You can’t prove a company has market power without explaining which market they have power in. According to the agency, the market for personal social networking services has three key attributes: First, a network has to be “built on a social graph that maps the connections between users and their friends, family, and other personal connections.” Second, it has to have features for users to interact with each other in a “shared social space,” like a news feed or group. Third, it has to allow users to look each other up. (Think about how you can search for someone by name on Facebook, but not in iMessage.)
The Green Bay Packers play in one of the tiniest media markets in the NFL, with a small but famously loyal fan base. It’s a key part of their charm. It’s also why it was so bewildering to discover that the single most-viewed URL on Facebook over the past three months, with 87.2 million views, belongs to an obscure site devoted to charging people to hang out with former Packers players.
That fact is one of several bizarre data points to emerge from Facebook’s first-ever “Widely Viewed Content Report.” The document is apparently an attempt to push back against the narrative that the platform is overrun with misinformation, fake news, and political extremism. According to data from its own publicly available analytics tool, CrowdTangle—data skillfully popularized by New York Times reporter Kevin Roose—the list of pages and posts with the highest engagement on the platform is heavily dominated by less-than-reputable right-wing publications and personalities like NewsMax and Dan Bongino, who vastly outperform more trustworthy mainstream publications.
Facebook has long argued that engagement doesn’t tell the whole story. A more accurate way to measure what’s popular on Facebook, the company’s executives say, is to look at total impressions, or “reach”—that is, how many people see a given piece of content rather than how many like or comment on it. The obvious problem with that argument is that, until Wednesday, Facebook had never shared any data on reach, making its claims impossible to verify. As Roose wrote last month, a proposal to make that data public ran into resistance within the company because it also might not make Facebook look so hot. As CrowdTangle CEO Brandon Silverman reportedly put it in an internal email, “Reach leaderboard isn’t a total win from a comms point of view.”
Now we have some idea of what Silverman may have meant.
The new report consists mostly of four Top 20 lists: the most viewed domains, links, pages, and posts over the last three months. (Facebook says it will release the reports quarterly.) The domains list contains mostly unsurprising results, including the likes of YouTube, Amazon, and GoFundMe—prominent websites that you’d expect to be posted a lot on Facebook. (Those results are not just unsurprising but unhelpful, since a link to the YouTube domain, say, could be for any one of literally billions of videos.) But number nine is the URL playeralumniresources.com—that Packers website. Things get even stranger in the Top 20 links ranking, where that URL comes in first place, meaning the homepage of Player Alumni Resources was somehow more popular on Facebook than every other site on the internet. The rest of the list contains similar surprises. In second place is a link to purehempshop.com; in fifth, with 51.6 million views, is reppnforchrist.com.
Is Player Alumni Resources, run by former Packers kicker Chris Jacke, quietly a Facebook juggernaut? Its official page has only 4,100 followers. Its posts get very few likes or comments. What’s going on here?
The answer: memes. From his personal account, which has more than 120,000 followers, Jacke posts a steady stream of low-rent viral memes that have nothing to do with the Packers, adding the URL of his business to the top of the post. We’re talking the likes of “Pick one cookie variety to live without,” or “Give yourself a point for each of these that you’ve done.” A post of a meme asking what word people use for soda (or pop, if you insist), for example, racked up more than 2 million interactions in June, according to CrowdTangle data. Jacke didn’t respond to requests for comment.
This seems to be the modus operandi of the other seemingly random members of the link leaderboard. The hemp store in second place, with 72.1 million views? That appears to be the handiwork of Jaleel White, best known for playing Steve Urkel on Family Matters. White, whose page has nearly 1.5 million followers, posts meme after recycled meme, each one graced with a link to a CBD product store.
Big Crypto has arrived. On August 10, following days of wrangling and furious tweeting, cryptocurrency enthusiasts, advocates, and entrepreneurs watched in horror as the US Senate approved a $1 trillion infrastructure bill, complete with an article that many fear might jeopardize the whole American crypto sector beyond repair. The controversial rule would require that “brokers” of transactions in digital assets—i.e., cryptocurrencies—report their customers to the Internal Revenue Service so they can be taxed.
The crypto crowd griped that the bill’s definition of “broker” was so broad it would potentially encompass miners, validators, and developers of decentralized applications—all of which, while playing pivotal roles in the functioning of a blockchain ecosystem, have no way of identifying their anonymous users.
Initially, it had looked like the bill’s language might be tweaked to exempt those categories, as a trio of senators put forth an amendment clarifying the “broker” term. Then a White House-backed amendment appeared, pushing for a less lenient clarification, exempting proof-of-work miners—which use an energy-intensive process to secure blockchains such as Bitcoin or Ethereum—but not many other categories, such as proof-of-stake validators, which carry out the same function without the energy burning. Just as a compromise position was being worked out, the Senate decided to pass the bill unamended. Any change will have to happen at a later stage—and it likely will, given the patent unenforceability of the bill as is.
On the face of it, it’s a drubbing for American crypto. But the narrative that has been doing the rounds is quite different: The infrastructure bill is a watershed moment in the history of cryptocurrency. The technology—at its core a crypto-anarchist, anti-bank, borderline anti-government manifesto disguised as code—has finally acquired that great marker of prestige: a lobby. The fact that some senators were ready to fight in crypto’s corner appears to show that the cryptocurrency industry is more than a gaggle of Twitter accounts and some blue-sky venture capitalists. Whatever the reason, it has influence, and—after the infrastructure bill saga—it will be ready to wield it even more deftly.
“We’re seeing the formalization, the maturing, of the crypto lobby, and this was the first coordinated effort that brought that to bear,” says Alex Brammer, vice president of business development at Luxor Tech, a bitcoin mining company. “Organizations like the Blockchain Association, the Texas Blockchain Council, or the Chamber of Digital Commerce are certainly going to continue their work.”
Cryptocurrency is usually, and lazily, described as a Wild West, but as a matter of fact the established businesses operating in the sector—from big mining enterprises to Wall Street–listed giants such as Coinbase—tend to crave regulation to define the boundaries of what is acceptable and what might get them into trouble. “Sophisticated players in this space welcome intelligent regulation. It provides clarity and predictability for large operations,” Brammer says. “It provides a set of rules of the road that allow large, publicly traded companies to make sure that they’re doing everything they can to be as viable and as profitable as possible going forward.”
But where does that leave the smaller, less established, less corporate players? Bitcoin—an asset owned and lionized by billionaires such as Mark Cuban and Elon Musk—has been growing since 2009 into an industry that carries heft and brand recognition. (Even Ted Cruz is waxing lyrical about it).
The much-contested amendment approved by the White House would have saved bitcoin while throwing much of crypto under the bus. Granted, when that plan emerged, the crypto lobby—or, at least, crypto-Twitter—rose as one against it. Jerry Brito, executive director of cryptocurrency trade group Coin Center thundered against the Senate’s attempt to pick “winners and losers,” while venture capitalist and crypto-ideologue Balaji Srinivasan said that the amendment would eventually open the door to a full-blown bitcoin ban. But it is worth wondering whether, in the long run, a rift might open between a Big Crypto clamoring for clear regulation to achieve peace of mind and the smaller actors of the cryptocurrency community, who might be less well equipped to meet the requirements that regulation would impose.
When Facebook said Tuesday that it was suspending the accounts of a team of NYU researchers, it made it seem like the company’s hands were tied. The team had been crowdsourcing data on political ad targeting via a browser extension, something Facebook had repeatedly warned them was not allowed.
“For months, we’ve attempted to work with New York University to provide three of their researchers the precise access they’ve asked for in a privacy-protected way,” wrote Mike Clark, Facebook’s product management director, in a blog post. “We took these actions to stop unauthorized scraping and protect people’s privacy in line with our privacy program under the FTC Order.”
Clark was referring to the consent decree imposed by the Federal Trade Commission in 2019, along with a $5 billion fine for privacy violations. You can understand the company’s predicament. If researchers want one thing, but a powerful federal regulator requires something else, the regulator is going to win.
Except Facebook wasn’t in that predicament, because the consent decree doesn’t prohibit what the researchers have been doing. Perhaps the company acted not to stay in the government’s good graces but because it doesn’t want the public to learn one of its most closely guarded secrets: who gets shown which ads, and why.
The FTC’s punishment grew out of the Cambridge Analytica scandal. In that case, nominally academic researchers got access to Facebook user data, and data about their friends, directly from Facebook. That data infamously ended up in the hands of Cambridge Analytica, which used it to microtarget on behalf of Donald Trump’s 2016 campaign.
The NYU project, the Ad Observer, works very differently. It doesn’t have direct access to Facebook data. Rather, it’s a browser extension. When a user downloads the extension, they agree to send the ads they see, including the information in the “Why am I seeing this ad?” widget, to the researchers. The researchers then infer which political ads are being targeted at which groups of users—data that Facebook doesn’t publicize.
Does that arrangement violate the consent decree? Two sections of the order could conceivably apply. Section 2 requires Facebook to get a user’s consent before sharing their data with someone else. Since the Ad Observer relies on users agreeing to share data, not Facebook itself, that isn’t relevant.
When Facebook shares data with outsiders, it “has certain obligations to police that data-sharing relationship,” says Jonathan Mayer, a professor of computer science and public affairs at Princeton. “But there’s nothing in the order about if a user wants to go off and tell a third party what they saw on Facebook.”
Joe Osborne, a Facebook spokesperson, acknowledges that the consent decree didn’t force Facebook to suspend the researchers’ accounts. Rather, he says, Section 7 of the decree requires Facebook to implement a “comprehensive privacy program” that “protects the privacy, confidentiality, and integrity” of user data. It’s Facebook’s privacy program, not the consent decree itself, that prohibits what the Ad Observer team has been doing. Specifically, Osborne says, the researchers repeatedly violated a section of Facebook’s terms of service that provides, “You may not access or collect data from our Products using automated means (without our prior permission).” The blog post announcing the account bans mentions scraping 10 times.
Laura Edelson, a PhD candidate at NYU and cocreator of the Ad Observer, rejects the suggestion that the tool is an automated scraper at all.
“Scraping is when I write a program to automatically scroll through a website and have the computer drive how the browser works and what’s downloaded,” she says. “That’s just not how our extension works. Our extension rides along with the user, and we only collect data for ads that are shown to the user.”
Bennett Cyphers, a technologist at the Electronic Frontier Foundation, agrees. “There’s not really a good, consistent definition of scraping,” he says, but the term is an odd fit when users are choosing to document and share their personal experiences on a platform “That just seems like it’s not something that Facebook is able to control. Unless they’re saying it’s against the terms of service for the user to be taking notes on their interactions with Facebook in any way.”
Ultimately, whether the extension is really “automated” is sort of beside the point, because Facebook could always change its own policy—or, under the existing policy, could simply give the researchers permission. So the more important question is whether the Ad Observer in fact violates anyone’s privacy. Osborne, the Facebook spokesperson, says that when the extension passes along an ad, it could be exposing information about other users who didn’t consent to sharing their data. If I have the extension installed, for instance, it could be sharing the identity of my friends who liked or commented on an ad.
