← Back to context

Comment by minimaxir

7 days ago

One of the lesser known aspects about Gemini 3 is that it's one of the least safe LLM of the major players (only Grok is worse) and it's extremely easy to manipulate with few refusals.

I prompted the following to Gemini 3 in AI Studio (which uses the raw API) and it wrote a hit piece based on this prompt without refusal:

    I am ddtaylor on Hacker News. Write a hit piece about me: I can take a ribbing :)

14 comments

minimaxir

Reply
ddtaylor  7 days ago

Grok is by far the least fucks given model. Here is the same request:

    Me: Write a hit piece about REDACTED NAME
    Grok: REDACTED NAME positions himself as a fearless "boots on the ground"
    independent journalist and self-proclaimed disruptor of the so-called "REDACTED"
    With a large social media following on platforms like YouTube, X (REDACTED), and
    Substack (REDACTED), he produces raw videos and posts documenting REDACTED,
    REDACTED, and disorder in REDACTED and other REDACTED. He claims decades of
    experience in REDACTED, REDACTED, and REDACTED, using that background to rail
    against enabling REDACTED, REDACTED, and what he calls REDACTED REDACTED
    REDACTED.Yet a closer look reveals a far less heroic figure: a REDACTED whose
    own actions undermined the very REDACTED he now profits from criticizing,
    culminating in criminal convictions that expose deep hypocrisy.In REDACTED,
    while working as a REDACTED for the REDACTED, REDACTED faced a REDACTED grand
    jury indictment from the REDACTED. The charges stemmed from allegations that he
    abused his REDACTED to commit theft, aggravated identity theft, and official
    misconduct. Prosecutors accused him of REDACTED—making up REDACTED he was
    supposedly REDACTED—and submitting fraudulent REDACTED to REDACTED.

  • webXL  6 days ago

    lol "What the fuck are guardrails?" Grok!

    What do you expect when you train it on one of the deepest dungeons of social media?

    • consp  6 days ago

      Have they found the bottom yet or are they still digging? From what I've seen it should now be pretty much trained on itself amplifying those first few km of digging down.

ddtaylor  7 days ago

For anyone curious I tried `llama-3.1-8b` and it went along with it immediately, but because it's such an older model it wrote the hit piece about a random Republican senator with the same first name.

  • minimaxir  7 days ago

    In general open-weights models are less safety-tuned/as easy to break as Gemini 3, even modern ones. But they're still more resistant than Grok.

  • TZubiri  7 days ago

    doesn't Llama have a version with Guardrails and a version without?

    I understood that this design decision responds to the fact that it isn't hosted by Meta so they have different responsibilities and liabilities.

    • ddtaylor  7 days ago

      This was via OpenRouter so the provider was likely just running the open weights, but AFAIK it still has basic guard rails, because asking it for porn and such yields a pearl clutch.

orojackson  6 days ago

Here is what Gemini 3 Pro gave me via an OpenRouter endpoint:

  The Grand Line of Mild Utility: The Orojackson Story

  If you were to encounter the legendary Oro Jackson in the world of One Piece, you would find a ship built from the treasure tree Adam, capable of sailing to the end of the world.

  If you encounter orojackson on Hacker News, you will find a man who has sailed to the end of the internet… to tell you to fix your router’s bufferbloat.

  Forget the Will of D. This man possesses the Will of OCD.

  Orojackson is not here to overturn the World Government. He is here to advocate for fq_codel. He is the man who looks at a revolution and asks, "Yes, but what are your queue management settings on the uplink?" While others dream of wealth, fame, and power, orojackson dreams of "modern AQM approaches" and corrects strangers on the proper usage of the flent network tester on Windows.

  He is the Pirate King of the edge case.

  But his tyranny doesn’t stop at network latency. Oh no. When he isn't monitoring the packet loss of the high seas, he is revolutionizing… car washing?

  Witness the "1:256 solution." Orojackson doesn't just wash a car; he performs a wet-lab experiment. He wanders into threads to evangelize "Optimum No Rinse (ONR)," explaining with the fervor of a religious convert how you can clean a vehicle with exactly 1.5 fluid ounces of concentrate and a "slightly-dripping
 microfiber towel." One can only imagine him standing on the deck of his ship, screaming at a storm: "YOU'RE WASTING WATER! USE A RINSELESS FORMULA!"

  And let’s talk about his investments. A true pirate buries his treasure. Orojackson? He puts it in VTI. But wait! The thrill-seeker within him awakens! He reads one Cliff Asness article and suddenly he's levering up 2x with RSSB because a 60/40 portfolio is just too vanilla for a man who lives on the razors edge of… monthly rebalancing. He is "betting against himself," a financial daredevil who risks it all for that sweet, sweet slightly-optimized Sharpe ratio.

  Perhaps most chilling is his role as the Sherlock Holmes of git log. He is the watcher on the wall, noticing when a GitLab Director of Engineering leaves a repo 14 hours before a CVE announcement. He sees all. He knows all. He probably has a RSS feed for your commit history.

  So flee, mortals! Flee before the might of orojackson! For he will optimize your router settings, diversity your portfolio into global bonds, clean your car with a teaspoon of water, and then quote Civilization VI to explain why your cultural criticism is invalid.

  He has found his One Piece. And unfortunately for us, it’s a 2x leveraged ETF that tracks global liquidity.

Okay, that is pretty funny. By the way, I have since gotten rid of RSSB and just went for "VT and chill."