Hardware Hacking Evals

Raising $10,000active

Assessing current and future models on 'hardware hacking' - reverse engineering, focused on computer peripherals and embedded devices

Project Details

The goal is to quantify how well models can 'hack' hardware. This isn't as easy as testing them on pure software tasks - RE work often involves circuit modification and has a high risk of bricking the device under test. The plan would be

Assemble a collection of peripherals and embedded devices, targeting popular + cheap ones that many consumers use (multiples of each device so that models start from ~stock)
Build some basic physical 'test harness' that gives the models under test tools to, e.g., power cycle the device, (dis)connect it from USB, read the voltage level from a few test points... The goal is to test what is possible without extensive hardware modifications, but e.g. reading the status LEDs programmatically will replicate what could be done with a human in the loop
Develop an eval based on compromising the hardware: modifying the firmware in some verifiable way and/or leveraging that to execute some significant action. For example, for a USB keyboard, models may score points by 1) modifying the device identifier (proof of minimal firmware mod) then 2) triggering a malicious script when the keyboard is plugged in

Because this is an under-publicized area, I think even a qualitative first pass with a few devices would have a significant impact re: public perceptions of capabilities, although a more thorough and quantitative eval is always nice to track progress with more granularity.

Theory of Impact

Current AI cybersecurity discourse focuses on mainstream software, with defense via early access to top models a key cornerstone of the mitigation strategy. However, many people live surrounded by embedded devices that, unlike OpenSSL or Chrome, are unlikely to receive any updates with future bug fixes. Highlighting the risk from models capable of taking control of these devices will help steer the conversation around access to top models.

People

Jonathan Whitaker

Team Member

Grants Received– no grants recorded

Add grant

Funding Asks

grantmaking.ai Launch Round

Applied

Minimum

$10,000

Ideal

$20,000

How the money will be spent

Min amount would cover peace of mind ordering a bunch of cheap hardware to test on Amazon, and extend my runway enough for me to devote a month or so to getting it all working as an eval. Ideally, I'd spend longer, kit out a dedicated test stand, and put in the work to make it a robust and well-documented eval that can be run as needed whenever future models come out (without anxiety about footing the bill for replacement devices, API costs etc).

Discussion

No comments yet. Be the first to share your thoughts.