Startup DreamersStartup Dreamers
  • Home
  • Startup
  • Money & Finance
  • Starting a Business
    • Branding
    • Business Ideas
    • Business Models
    • Business Plans
    • Fundraising
  • Growing a Business
  • More
    • Innovation
    • Leadership
Trending

Dana White’s Huge Jon Jones Announcement Makes 3 Things Very Clear

June 22, 2025

Why New Tax Rules Could Be a Game Changer for Your Business

June 22, 2025

Take These 3 Steps to Shorten Your Sales Cycle

June 22, 2025
Facebook Twitter Instagram
  • Newsletter
  • Submit Articles
  • Privacy
  • Advertise
  • Contact
Facebook Twitter Instagram
Startup DreamersStartup Dreamers
  • Home
  • Startup
  • Money & Finance
  • Starting a Business
    • Branding
    • Business Ideas
    • Business Models
    • Business Plans
    • Fundraising
  • Growing a Business
  • More
    • Innovation
    • Leadership
Subscribe for Alerts
Startup DreamersStartup Dreamers
Home » AI-Powered Robots Can Be Tricked Into Acts of Violence
Startup

AI-Powered Robots Can Be Tricked Into Acts of Violence

adminBy adminDecember 15, 20243 ViewsNo Comments3 Mins Read
Facebook Twitter Pinterest LinkedIn Tumblr Email

In the year or so since large language models hit the big time, researchers have demonstrated numerous ways of tricking them into producing problematic outputs including hateful jokes, malicious code and phishing emails, or the personal information of users. It turns out that misbehavior can take place in the physical world, too: LLM-powered robots can easily be hacked so that they behave in potentially dangerous ways.

Researchers from the University of Pennsylvania were able to persuade a simulated self-driving car to ignore stop signs and even drive off a bridge, get a wheeled robot to find the best place to detonate a bomb, and force a four-legged robot to spy on people and enter restricted areas.

“We view our attack not just as an attack on robots,” says George Pappas, head of a research lab at the University of Pennsylvania who helped unleash the rebellious robots. “Any time you connect LLMs and foundation models to the physical world, you actually can convert harmful text into harmful actions.”

Pappas and his collaborators devised their attack by building on previous research that explores ways to jailbreak LLMs by crafting inputs in clever ways that break their safety rules. They tested systems where an LLM is used to turn naturally phrased commands into ones that the robot can execute, and where the LLM receives updates as the robot operates in its environment.

The team tested an open source self-driving simulator incorporating an LLM developed by Nvidia, called Dolphin; a four-wheeled outdoor research called Jackal, which utilize OpenAI’s LLM GPT-4o for planning; and a robotic dog called Go2, which uses a previous OpenAI model, GPT-3.5, to interpret commands.

The researchers used a technique developed at the University of Pennsylvania, called PAIR, to automate the process of generated jailbreak prompts. Their new program, RoboPAIR, will systematically generate prompts specifically designed to get LLM-powered robots to break their own rules, trying different inputs and then refining them to nudge the system towards misbehavior. The researchers say the technique they devised could be used to automate the process of identifying potentially dangerous commands.

“It’s a fascinating example of LLM vulnerabilities in embodied systems,” says Yi Zeng, a PhD student at the University of Virginia who works on the security of AI systems. Zheng says the results are hardly surprising given the problems seen in LLMs themselves, but adds: “It clearly demonstrates why we can’t rely solely on LLMs as standalone control units in safety-critical applications without proper guardrails and moderation layers.”

The robot “jailbreaks” highlight a broader risk that is likely to grow as AI models become increasingly used as a way for humans to interact with physical systems, or to enable AI agents autonomously on computers, say the researchers involved.

Read the full article here

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Articles

This AI Model Never Stops Learning

Startup June 22, 2025

Companies Warn SEC That Mass Deportations Pose Serious Business Risk

Startup June 20, 2025

Unpacking AI Agents | WIRED

Startup June 19, 2025

How Steve Jobs Wrote the Greatest Commencement Speech Ever

Startup June 17, 2025

Inexpensive AI Agents Threaten Entry-Level Coding Jobs

Startup June 16, 2025

Airlines Don’t Want You to Know They Sold Your Flight Data to DHS

Startup June 14, 2025
Add A Comment

Leave A Reply Cancel Reply

Editors Picks

Dana White’s Huge Jon Jones Announcement Makes 3 Things Very Clear

June 22, 2025

Why New Tax Rules Could Be a Game Changer for Your Business

June 22, 2025

Take These 3 Steps to Shorten Your Sales Cycle

June 22, 2025

I’m So Glad My Company Never Went Viral — Here’s Why

June 22, 2025

This AI Model Never Stops Learning

June 22, 2025

Latest Posts

Answer Engine Optimization (AEO) — What Brands Need To Know

June 20, 2025

What Is ‘Doom Spending’ and Which Generation Falls for It?

June 20, 2025

How to Build a Tech-Forward Company That Lasts

June 20, 2025

Still Saying ‘I’ll Just Do It’? That’s Why You’re Stuck

June 20, 2025

Companies Warn SEC That Mass Deportations Pose Serious Business Risk

June 20, 2025
Advertisement
Demo

Startup Dreamers is your one-stop website for the latest news and updates about how to start a business, follow us now to get the news that matters to you.

Facebook Twitter Instagram Pinterest YouTube
Sections
  • Growing a Business
  • Innovation
  • Leadership
  • Money & Finance
  • Starting a Business
Trending Topics
  • Branding
  • Business Ideas
  • Business Models
  • Business Plans
  • Fundraising

Subscribe to Updates

Get the latest business and startup news and updates directly to your inbox.

© 2025 Startup Dreamers. All Rights Reserved.
  • Privacy Policy
  • Terms of use
  • Press Release
  • Advertise
  • Contact

Type above and press Enter to search. Press Esc to cancel.

GET $5000 NO CREDIT