Corbucci
-
Postdoc @ FBK, Trento, Machine Translation Unit
-
Podcaster @ PointerPodcast
-
Co-founder @ Superhero Valley
-
Co-founder @ Pisa.dev
- lucacorbucci.me
everything.
it’s safe?
fascinating research
problems..."
Learning
real sentences.
biases.
public bias.
stay in the data.
How it
starts
The model doesn't invent prejudice. Skewed training data becomes skewed predictions.
What it
damages
Decisions that should be merit-based become proxies for race, gender, or wealth. This is automated and it happens at a scale no human bias ever could.
What to
do
Disaggregate your metrics. A model can be accurate and still unfair. Audit your model by subgroup.
accountability.
How it
starts
Model gives a score or answer, but no reason. "Why?" becomes a guess, not an explanation.
What it
damages
No debugging, no appeal, no meaningful audit. If nobody can inspect the decision, nobody can defend it.
What to
do
Use interpretable models when possible. Use XAI tools to extract insights from your model. If you can't explain it, you can't trust it.
more than they should.
How it
starts
Models don't just learn patterns, they memorize. Rare sequences, personal records, credentials get encoded directly into model weights.
What it
damages
The model becomes an unintended data store. Query it the right way and it reproduces content it was never meant to expose.
What to
do
Scrub sensitive data before training. Apply differential privacy to guarantee protection. Red-team your model.
Language
Models
real liability.
brand damage.
get real people's data.
sound right.
How it
starts
The model predicts the most plausible next token, not the most truthful one. It has no mechanism to know what it doesn't know.
What it
damages
Fabricated citations, invented policies, and wrong technical answers delivered with full confidence.
What to
do
Ground outputs on verified sources. Never let model output reach a user without a trust boundary between generation and delivery.
breaks the guardrails.
How it
starts
Safety rules are part of the model's behavior, not hard-coded limits. Attackers use roleplay, indirection, or adversarial prompts to bypass them.
What it
damages
The model reveals content it was supposed to refuse: harmful instructions, toxic outputs, hidden prompts, or restricted information.
What to
do
Use layered protections: stronger alignment, adversarial training, output monitoring, and continuous red-teaming.
becomes the attacker.
How it
starts
A malicious instruction hidden inside a PDF gets processed as if it was part of the prompt.
What it
damages
The LLM silently changes behavior across the workflow and outputs content it was not designed to produce.
What to
do
Treat external content as untrusted input. Isolate them from the user prompts.
but they don't close the gap.
Everything gone.
one silent breach.
Public hit piece.
Your files, exfiltrated.
attack surface.
Blurred
trust
Data and instructions share the same context. Any document, email, or web page the agent reads can hijack what it does next.
Real
actions
Agents don't just talk, they execute. A mistake isn't a wrong answer, it's a deleted database, a leaked file, a published post.
No
undo
By the time a human notices, the action is already done. Speed and autonomy collapse the window for oversight.
but agents still act.
Least
privilege
Give the agent only the permissions the task truly needs.
Minimal
footprint
Prefer reversible actions and confirm before anything irreversible.
Human in
the loop
Put a person in front of high-stakes or destructive actions.
Sandboxing
Isolate agent execution so the blast radius stays bounded.
Audit
logs
Record every tool call with inputs, outputs, and timestamp.
Test
adversarially
Assume a clever prompt will still try to cross the boundary.
Same pattern.
you wouldn’t do.
Questions?
