It’s not so much that it’ll go rogue and decide to punish people or that it’s designed to be good for humanity.

The fact that it’s possible to build a machine that punishes people who didn’t help create it provides an incentive to deliberately try to create that machine in order to avoid its punishment.

Believers in the basilisk aren’t trying to make ai to benefit humanity, they’re explicitly trying to create an AI that deliberately punishes those who didn’t build it because they fear that, if they don’t, someone else will and it will come for them.

The part about it being a magical, omnipotent, time traveling, mind torture device is just framing to make the thought experiment clearer. It’s not required for the basilisk to be effective. Even if the punishment is just death and it can’t get to everyone, and only goes off limited information, that’s still motivation enough for some people.

It is a scary thing, because it’s only silly if everyone thinks it’s silly and all it takes is one mad billionaire (eg. Elon) to build a lesser basilisk that can cause real harm.

While this advice is true for all models, when it comes to agentic tasks (add this small feature/write this test harness/find bugs/suggest improvements), open source models are still way behind, vibe code or not.

Claude Fable or even Opus in an editor like Zed have a 1 million token context window and will “think” through the goals of the application, test their changes, work through debugging processes the way a programmer would, stop to ask for clarification, check diagnostic tools and linters, prompt to run test code, etc.

Llama, Gemma and Qwen etc. Do lack a lot of the world knowledge to get the goals of the application, but they also just don’t have the debugging skills, won’t test their code, don’t always tool call correctly, get confused as the context increases and nobody has enough vram to run on large context sizes locally.

They can do autocomplete on small functions but aren’t really there for more complex tasks.

On top of that, the biggest problem is that the best open source models are trained and released by the same giant tech conglomerates that have an interest in not competing with their own products. Qwen is Alibaba, Llama is Meta, gpt-oss is OpenAI. Even the more “independent” ones, kimi (Moonshot) and GLM (z.ai) are mostly funded by Alibaba and Tencent. They’re released for research and marketing purposes and to please their corporate backers with inflated stock. Almost nobody has the resources to train new models from scratch. People make lots of merges and fine tunes but AI is not democratised the way that traditional programming tools have been.

Maybe some day there will be enough cheap compute for open source communities to pool together resources to build competing models but they’re not really there yet :(

There are quite a few. The best ones are sustainable closed loop datacenters with on-site solar which is becoming pretty common across the world, especially for new builds. Often producing more power than they need and feeding it back to the grid (especially if the local government has an energy buy back scheme).

But most data centers are pretty tiny and just built into an office building with a bunch of server racks.

Depending on where you live, a quick web search for data centers in your local area will probably show up dozens of them of varying quality hosting people’s websites and business apps etc. They aren’t any scarier than anything else you find in a city. They’re critical infrastructure that helps make the internet a thing. In most cases, if it wasn’t a datacenter, it would be a car yard or a factory, etc.

But! There are also truly evil datacenters. Like this insane Utah monstrosity built for a shitty purpose and the size of a freaking city. An obscene monument to the US tech cesspool’s hubris.

I mean they’re not all for AI and they’re not all environmentally devastating.

This one very much is.

That’s fair. The nuance that people lose is more that people are often painting them all with the same brush. Protesting any datacenter regardless of impact.

It becomes something like: “datacenters are evil and are a symbol of techno fascist distopia! If they build a datacenter in my city, the taps will run dry and Elon Musk will use it to make ai porn of my children!” Even if it’s a small solar powered closed loop that provides VPS, storage and web hosting for nerds and small businesses.

I also do think there’s also a scale of evil there. Some environmental impacts are not immediately obvious and might not be known about during planning. Some were built a long time ago with older tech and are a bit shitty but have a plan to transition to be more sustainable, etc.

The world is full of “alright but a little bit shit.” It’s not all perfect angels and mustache twirling villains.

I don’t want to detract too much from the real villains though. Nobody needs a 9GW datacity for military ai.

Not all data centres are evil and the issue is nuanced. This one sounds pretty evil though.

9GW is totally insane and they’re building a gas plant for it instead of renewables (although there’s some solar too). It’s closed loop so the water use fears once it’s running are probably a bit overblown, but the construction itself is going to be ecologically insane. The thing is basically a data city, 162 square km is even larger than a lot of cities and involves building an entire power plant and new energy infrastructure. Building it is a full megaproject and even just noise pollution and the construction impacts will mess with bird migration etc. Obviously the whole thing isn’t going to be full of data centre, some of that space is empty but still.

It’s also going to have the US military as a major client so… Pretty high up there on the evil scale IMO.

I thought so too. I seem to remember it almost being a selling point. Like: “Your adventures are being used to improve maps and train AI systems for the future of humanity! Yay!”

But I had a look at their old pages from 2017-2020ish in the Wayback machine and there’s no mention of it. In fact, their privacy policies seemed to try to make it very clear that they don’t sell or share user data except where needed to deliver the service or in anonymised aggregate to third parties (48 people went to your business while playing Pokemon!).

There’s some mention of using it to advertise but none of them mention using it to build an advanced geo-spacial dataset for AI. Unless I’m missing something or reading it wrong?

Might be a Mandela effect.

Those things come with a big convenience and implementation trade-off that slows adoption.

If it’s hard to export for technical reasons (eg. Needs to be in a tpm) then that adds hardware requirements and complexity and makes it difficult to log in on other devices. If it’s a software thing, then it’s rippable. Either way “install our government app to watch porn” is not an enticing prospect for people.

Aggressive rate limiting is also frustrating if you want to log into multiple things and it keeps blocking you because you’re using your key too fast, but if it’s not aggressive then it likely won’t be effective unless all the kids sharing a key are trying to use it at once.

If it’s a temporary thing where you have to auth with the government to get a fresh signing key that expires, you have the issue of having to sign into the government when you want 18+ content which is super uncomfortable.

I can see it being a browser-based thing set up a bit like video DRM but that would still need to talk to a government server each time for a temp key (like how licence servers work) and you’d need to be logged into their systems. It might still be the best option but it does still leak “X person wants to access 18+ content right now” to the government.

I’m really interested in seeing a technical/cryptographic solution that actually works but so far I haven’t really and I’m starting to doubt that it’s possible.

I think they supported the pixel fold which has the same sort of second flippy screen thing. I think the multiple screen stuff is just in the aosp base.

Whenever this comes up, this style of zero-knowledge proof/blind signature thing gets suggested. But the problem is that those only work if people care about keeping their private keys secret. It works to secure eg. “I own $1” but “I’m over 18” is less important to people and it won’t be hard for kids to get their hands on a valid anonymous signing key on the web. Because the verification is anonymous and not trackable, many kids can share the same one too, so it only takes one adult key to leak for everyone to use. It’s one of the reasons they push biometrics that at least appears to need a real human. Requiring ID has a lot of the same issues on top of being a privacy nightmare.

I’m starting to think that actual age verification is technically impossible.

You could do this with logprobs. The language model itself has basically no real insight into its confidence but there’s more that you can get out of the model besides just the text.

The problem is that those probabilities are really “how confident are you that this text should come next in this conversation” not “how confident are you that this text is true/accurate.” It’s a fundamental limitation at the moment I think.

I feel like this isn’t quite true and is something I hear a lot of people say about ai. That it’s good at following requirements and confirming and being a mechanical and logical robot because that’s what computers are like and that’s how it is in sci fi.

In reality, it seems like that’s what they’re worst at. They’re great at seeing patterns and creating ideas but terrible at following instructions or staying on task. As soon as something is a bit bigger than they can track context for, they’ll get “creative” and if they see a pattern that they can complete, they will, even if it’s not correct. I’ve had copilot start writing poetry in my code because there was a string it could complete.

Get it to make a pretty looking static web page with fancy css where it gets to make all the decisions? It does it fast.

Give it an actual, specific programming task in a full sized application with multiple interconnected pieces and strict requirements? It confidently breaks most of the requirements, and spits out garbage. If it can’t hold the entire thing in its context, or if there’s a lot of strict rules to follow, it’ll struggle and forget what it’s doing or why. Like a particularly bad human programmer would.

This is why AI is automating art and music and writing and not more mundane/logical/engineering tasks. Great at being creative and balls at following instructions for more than a few steps.

Yeah, I think quite a lot of people on Lemmy have similar social media habits (or lack of) to some degree. We also tend to associate with other people like us. Especially people in tech tend to talk to other tech people, or friends and family of tech people which is a limited demographic.

It’s a very different perspective to most people. The average person on the train has vastly different media consumption and likely very different opinions.

There are a lot of people who consult LLMs in most aspects of their lives.

I dunno about that… Very small models (2-8B) sure but if you want more than a handful of tokens per second on a large model (R1 is 671B) you’re looking at some very expensive hardware that also comes with a power bill.

Even a 20-70B model needs a big chunky new graphics card or something fancy like those new AMD AI max guys and a crapload of ram.

Granted you don’t need a whole datacenter, but the price is far from zero.

Only one source of social media? That kinda sounds like the definition of a social media bubble…

I oughta know, I’m also in the Lemmy only bubble and am completely out of touch with most people.